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PREFACE 


This  report  has  been  prepared  by  Wilson  Hill  Associates, 

Inc.  for  the  Transportation  Systems  Center's  (TSC)  Service 
Assessment  Division  in  support  of  the  Urban  Mass  Transportation 
Administration's  (UMTA)  Impact  Assessment  Program  sponsored  by 
The  Office  of  Methods  and  Support.  UMTA  is  cooperating  with 
and  supporting  the  transit  industry's  efforts  to  improve  the 
productivity  and  quality  of  telephone/information  marketing 
services  to  the  public.  The  principal  focus  of  this  support  is 
on  the  sponsorship  of  Automated  Transit  Information 
Systems  (ATIS)  technology  demonstrations.  TSC,  in  support  of 
UMTA's  program,  is  evaluating  ATIS  demonstrations  in  Los  Angeles 
and  Washington,  D.C.  This  report,  an  interim  product  in  an 
ongoing  ATIS  impact  assessment  program,  documents  the  Washington 
D.C.  demonstration. 

While  specifically  intended  for  the  information  of  TSC  and 
the  management  staff  of  the  Washington  Metropolitan  Area  Transit 
Authority,  this  report  will  also  be  of  interest  to  others  in 
the  transit  public  information/marketing  community  who  wish  to 
study  applications  of  computer-aided  information  retrieval 
systems. 

The  evaluations  described  in  this  report  were  conducted 
under  the  management  of  Mr.  Robert  Furniss  and  Mr.  Robert  Phillips, 
Project  Managers  for  Wilson  Hill  Associates.  Mr.  Furniss  was 
Project  Manager  through  February,  1981  when  he  left  Wilson 
Hill.  On  this  date,  Mr.  Robert  Phillips  assumed  project  manage- 
ment responsibilities.  For  the  Washington  D.C.  evaluation, 

Mr.  Furniss  directed  most  "pre- implementation"  data  collection 
activities,  while  Mr.  Phillips  managed  the  project  through  the 
"post- implementation"  phase  and  wrote  this  evaluation  report. 
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EXECUTIVE  SUMMARY 


A telephone  information  service  is  one  tool  frequently 
employed  by  transit  authorities  as  part  of  their  overall 
public  marketing  strategy.  Such  a telephone  information 
service  has  several  distinct  advantages  as  compared  with 
alternative  media  such  as  printed  service  schedules  or  maps. 
First,  it  is  easy  to  use  and  is  accessible  to  a substantial 
portion  of  the  transit-riding  public.  Second,  it  can  be 
frequently  updated  to  reflect  changes  in  routes  and  schedules, 
breakdowns,  special  services,  etc.  Finally,  and  most 
importantly,  it  can  be  personalized  to  meet  the  needs  of  the 
individual  customer. 

At  most  transit  authorities,  telephone  information 
operators  employed  by  the  authority  have  traditionally 
provided  information  to  callers  by  referencing  large  indexed 
volumes  of  schedules,  headway  sheets,  maps,  route  descriptions, 
etc.  located  at  their  work  stations.  This  process  can  be 
time-consuming,  tedious,  and  heavily  labor-intensive.  It  is 
therefore  quite  costly  for  transit  properties  to  maintain  a 
telephone  information  operation  in  proportion  to  consumer 
demand  for  this  information.  In  most  instances,  tradeoffs 
are  made  in  favor  of  cost  reductions  which  result  in  infor- 
mation service  shortfalls  of  up  to  30  percent  during  peak 
demand  periods. 

The  Urban  Mass  Transportation  Administration  (UMTA)  has 
over  the  past  eight  years  been  implementing  a research  and 
development  program  aimed  at  determining  the  feasibility  of 
applying  state-of-the-art  computer  technology  to  the  provision 
of  transit  information.  This  program  is  referred  to  as  the 
Automated  Transit  Information  System  (ATIS)  program.  Its 
objective  is  the  innovative  introduction  of  automation  to  the 
job  of  the  telephone  information  operator  in  order  to  reduce 
costs,  increase  productivity,  or  improve  service  provided. 
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Recent  advances  in  computer  technology  have  enhanced  the 
potential  for  cost-effective  automation  of  data  retrieval 
functions  routinely  performed  by  transit  telephone  information 
operators.  Certain  aspects  of  these  functions  make  them 
well-suited  for  computer  implementation.  First,  they  are 
highly  repetitive;  most  caller  requests  can  be  classified 
into  four  distinct  types,  each  requiring  consultation  of 
specific  reference  data.  Second,  they  require  considerable 
memorization  by  the  operator  of  service  area  geography,  routes, 
schedules,  etc.  These  characteristics  can  be  mapped  onto  a 
coordinate  grid  and  stored  in  the  computer's  memory.  Third, 
information  operators  often  optimize  certain  trip  parameters 
for  the  caller  (such  as  travel  time  or  walking  distance) . A 
computer's  capability  to  repeatedly  compare  alternatives  can 
be  exploited  by  translating  operator  comparison/optimization 
logic  into  logical  computer  program  routines.  Fourth,  it  is 
important  that  operator-supplied  information  exhibit  a high 
degree  of  consistency.  Computer  programming  logic  ensures 
that  the  same  query  will  always  elicit  the  same  response. 
Finally,  due  to  high  caller  demand,  information  operators 
must  work  as  rapidly  as  possible.  The  high  speed  calculating 
characteristics  of  a computer  guarantee  that  automated  pro- 
cessing of  a given  caller  request  will  involve  a much  more 
thorough  investigation  of  alternative  responses  than  time 
limitations  would  permit  using  traditional  manual  retrieval 
methods. 

Thus,  the  application  of  computer  logic,  memory  and 
speed  to  these  data  retrieval  functions  raises  the  potential 
for  improving  an  information  operator's  job  in  a variety  of 
ways.  These  improvements  could  also  produce  benefits  not  only 
for  the  agents,  but  also  for  the  transit  authority  management 
and  the  general  public  as  well.  Among  these  hypothesized 
benefits  are: 
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• improved  productivity  in  terms  of  agent  calls  per 
hour  due  to  faster  data  retrieval; 

• increased  reliability  and  consistency  of  agent 
responses; 

• a reduction  in  training  time  for  new  agents; 

• improved  capability  to  rapidly  incorporate  changes 
in  transit  agent  reference  data; 

• enhancement  of  the  agent's  overall  job  satisfaction; 

• expanded  capability  to  gather  statistics  concerning 
individual  agent  and  Information  Section  performance; 
and 

• use  of  the  transit  system  data  base  by  otlier  departments 
as  a planning  source. 

Computer  assisted  data  retrieval  changes  the  information 
agent's  job  in  a variety  of  ways.  ATIS  operation  involves  an 
agent's  retrieving  desired  information  through  the  use  of  a 
remote  interactive  computer  terminal  located  at  the  agent's 
work  station.  The  system  is  activated  by  the  agent's  entering 
caller  query  data  on  the  keyboard  and  selecting  the  proper 
functions  to  produce  the  desired  response.  Following  processing, 
which  takes  a matter  of  seconds,  the  computer  responds  with 
several  appropriate  alternatives.  The  agent  then  evaluates 
these  responses  in  light  of  the  query  and  selects  the  one 
best  suited  to  the  caller's  needs. 

This  report  documents  a number  of  data  collection  and 
analysis  efforts  carried  out  by  Wilson  Hill  Associates,  Inc. 
for  the  U.S.  Department  of  Transportation's  Transportation 
Systems  Center  (TSC) , Service  Assessment  Division.  The  TSC, 
as  part  of  UMTA's  Impact  Assessment  Program,  is  evaluating 
ATIS  demonstration  deployments  in  Washington,  D.C.  and  in  Los 
Angeles.  The  ATIS  at  the  Washington  Metropolitan  Area  Transit 
Authority  (WMATA) , known  as  the  Automated  Information  Directory 
System  (AIDS) , is  the  subject  of  this  evaluation  report. 
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The  Information  Section  of  the  Office  of  Marketing  at 
WMATA  is  one  of  the  largest  telephone  transit  marketing  centers 
in  the  nation  and  a prime  candidate  for  ATIS  implementation. 

Its  57  operators  answer  caller  queries  about  transit  service 
in  three  shifts,- 18  hours  a day,  365  days  a year.  On  an 
average  day  between  5,000  and  10,000  calls  are  received  by 
the  Section.  During  peak  demand  periods,  operators  answer 
over  800  calls  per  hour;  for  years  the  WMATA  telephone  infor- 
mation number  has  been  the  busiest  one  in  the  Washington, 

D.C. , Metropolitan  area.  Despite  this  effort,  over  30%  of 
all  incoming  calls  are  "lost”  in  peak  periods;  that  is, 
callers  placed  in  a holding  queue  by  telephone  answering 
equipment  hang  up  before  ever  reaching  an  operator  or 
receiving  any  information. 

In  undertaking  this  evaluation,  Wilson  Hill's  objectives 
were  twofold:  first,  to  ascertain  the  effects  of  automation 
on  agent  performance  and  job  satisfaction;  and  second,  to 
determine  the  extent  to  which  the  system  demonstration  had 
achieved  its  intended  benefits,  as  stated  above. 

In  this  effort,  large  amounts  of  quantitative  and 
qualitative  data  were  collected.  Because  this  evaluation 
began  while  AIDS  was  still  undergoing  design  checkout  and 
system  acceptance  testing,  most  data  were  collected  using  a 
"before-and-af ter " methodology  for  comparison  purposes. 
Quantitative  data  included  statistics  on  caller  and  Information 
Section  performance  as  recorded  by  the  Automatic  Call  Distri- 
bution System  (which  keeps  track  of  incoming  calls) , performance 
data  of  various  types  collected  by  evaluation  personnel  moni- 
toring the  operators  at  their  work  stations,  and  summaries  of 
all  computer  transactions  as  recorded  on  the  AIDS  Daily  Log 
File.  Qualitative  data  were  gathered  through  opinion  surveys 
administered  to  the  information  agents  both  before  and  after 
the  system  implementation  and  through  interviews  with  WMATA 
management  and  supervisory  personnel,  as  well  as  with  the 
agents  themselves. 


xiv 


A careful  review  of  these  quantitative  and  qualitative 
data  leads  to  generally  positive  findings  concerning  the 
overall  performance  of  the  AIDS  in  its  demonstration.  While 
the  implementation  process  was  somewhat  extended  due  to  a 
variety  of  unforeseen  problems,  the  data  collected  show  that 
the  system  is,  in  most  cases,  currently  functioning  as  originally 
intended  and  even  exceeding  expectations  in  some  areas.  System 
performance  in  achieving  a number  of  different  implementation 
objectives  is  discussed  in  the  following  sections. 

• Agent  Productivity;  Data  collected  during  the  course  of 
the  evaluation  lead  to  conflicting  conclusions  regarding 
system  influences  on  operator  productivity.  Agents  were 
evenly  divided  on  the  after-implementation  questionnaire 
regarding  AIDS'  influence  on  call  counts,  and  a number 
of  them  complained  about  "slowness"  of  system  response. 
Furthermore,  telephone  answering  equipment  data  revealed 
a decrease  overall  in  Information  Section  productivity 
in  the  post- implementation  period.  In  contrast,  agents 
were  almost  unanimous  in  the  opinion  that  AIDS  made 
their  jobs  easier,  and  routine  performance  tests  conducted 
by  Information  Section  management  reveal  that  operators 
using  AIDS  are  capable  of  answering  calls  at  the  rate  of 
30  to  40  per  hour,  a 30%  increase  over  their  normal 
manual  speed. 

WMATA  supervisory  and  management  personnel  were 
able  to  generally  explain  the  causes  underlying  these 
conflicting  results.  It  appears  that  with  current 
system  function  configurations  and  response  speed, 
questions  about  schedules  are  still  handled  fastest 
manually;  this  fact  is  supported  by  call  monitoring 
data.  Schedule  questions  make  up  about  one-quarter  of 
all  incoming  calls.  Thus,  agents  who  use  the  computer 
for  nearly  every  call  are  actually  less  productive  than 
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those  who  use  it  more  selectively.  Varying  agent  experience 
levels  are  a factor  contributing  to  post- implementation 
productivity  declines  since  newer  agents  tend  to  rely  on 
computer  or  manual  references  more  than  their  more 
experienced, counterparts,  who  rely  more  on  memory.  The 
Information  Section  experienced  considerable  turnover 
during  system  implementation.  Supervisory  personnel 
indicated  that  very  experienced  operators  (having  10  or 
more  years  on  the  job)  are  able  to  answer  up  to  75%  of 
their  calls  from  memory.  These  operators  tend  to  use 
AIDS  only  for  calls  requiring  construction  of  long  or 
complex  itineraries.  This  job  is  tedious,  but  is  well- 
suited  to  the  computer's  abilities  to  repeat  pre- 
programmed comparisons  and  to  optimize  certain  variables. 

It  is  not  surprising,  then,  that  more  experienced  operators 
would  feel  that  using  the  computer  would  slow  them  down 
in  most  instances,  but  would  still  be  of  the  opinion 
that  it  made  their  job  easier  (for  the  more  complex 
queries) . 

Supervisory  personnel  indicated  that  they  were 
initially  discouraged  by  agent  call  counts  following 
AIDS  training.  When  they  made  it  clear  to  the  agents 
that  using  AIDS  was  not  necessarily  the  most  useful  nor 
the  most  efficient  way  to  respond  to  every  call,  pro- 
ductivity increased  considerably,  usually  to  levels 
above  the  previous  average  for  manual  data  retrieval. 
Management  and  supervisors  both  felt  that  a 50-50  balance 
between  computer  and  manual  data  retrieval  produced  the 
highest  overall  call  count  for  the  average  agent.  Call 
monitoring  results  show  that  this  balance  has  not  yet 
been  achieved;  agents  are  using  AIDS  for  approximately 
25-30%  of  all  calls. 

Response  Accuracy  and  Consistency;  Agents  responding  to 
the  after-implementation  survey  indicated  by  overwhelming 
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margins  that  AIDS  responses  were  at  least  as  accurate 
and  had  a greater  level  of  detail  than  responses  retrieved 
manually.  Supervisory  personnel  stated  that  the  problem 
of  misinformation,  while  affecting  only  a small  percentage 
(1-2%)  of  all  calls  answered,  was  greatest  with  inex- 
perienced operators  using  manual  materials  only.  Super- 
visors indicated  that  they  felt  much  more  comfortable 
about  response  accuracy  when  these  operators  consulted 
AIDS. 

A feature  of  AIDS  that  helps  ensure  the  near  100% 
accuracy  of  its  data  base  is  the  built-in  error-reporting 
function.  If  an  operator  receives  a response  that  he/she 
deems  questionable,  activating  the  reporting  function 
will  cause  the  response  to  be  listed,  along  with  the 
agent's  ID  number,  on  a daily  "trouble  report".  A review 
of  this  report  by  agents  and  management  helps  to  locate 
inconsistencies  or  errors  in  the  data  base,  as  well  as 
to  identify  areas  in  which  an  agent's  training  might  be 
deficient.  It  cannot  be  overemphasized  that  this  enhance- 
ment process  is  an  interactive  one  in  which  agents  and 
Information  Section  management  are  active  participants. 

This  interaction  not  only  results  in  greater  accuracy  of 
the  data  base  and  a more  workable  system,  but  also  leads 
to  increased  confidence  on  the  part  of  the  agents  that 
they  are  personally  responsible  for  system  accuracy. 

Agent  Training:  Interviews  with  Information  Section 

management  indicated  that  the  anticipated  benefit  of 
training  time  reductions  for  new  operators  does  not 
appear  to  be  achievable,  based  on  results  with  four  new 
agents.  These  agents  received  a two-week  course  which 
included  computer  practice,  communications  skills  develop- 
ment, and  some  rudimentary  background  in  use  of  manual 
references  (they  already  possessed  typing  skills).  They 
were  not  given  the  intensive  geographic  and  transit 
system  practice  which  forms  a major  part  of  the  traditional 
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6-week  manual  training  period.  After  two  weeks,  these 
agents  were  producing  "acceptable"  journeyman-level  call 
counts  of  80-90  per  day,  and  it  seemed  as  though  the 
training  reduction  benefit  had  been  achieved. 

Reports  from  supervisory  personnel,  however,  soon 
produced  evidence  to  the  contrary.  Without  the  more 
extensive  geographic  and  transit  system  practice,  these 
new  agents  were  unable  to  provide  details  to  callers 
concerning  landmarks  to  use  as  reference  points  during 
their  journeys.  Even  worse,  they  were  unable  to  make 
interpretive  judgements  between  the  four  alternative 
itinerary  responses  routinely  produced  by  AIDS.  In  the 
opinion  of  the  supervisors,  they  became  too  dependent  on 
the  computer  as  a "crutch"  for  some  questions  such  as  fare 
and  schedule  requests,  which  other  agents  would  have  handled 
more  quickly  either  manually  or  from  memory.  To  remedy 
these  observed  deficiencies  in  their  performance,  these 
agents  were  later  given  additional  training  emphasizing 
map  work  and  manual  schedule  practice.  VJith  this  additional 
training,  they  were  better  able  to  picture  routes,  inter- 
sections, and  landmarks,  and  achieved  full  proficiency 
(150-170  calls  per  day). 

The  net  result  of  this  experience  was  to  point  out 
that  agents  with  AIDS  training,  but  lacking  a firm  founda- 
tion in  manual  practice  and  geographic  data  manipulation, 
are  of  marginal  value  given  the  present  system  configuration. 
While  these  operators  could  be  used  as  a stopgap  measure 
during  periods  of  critical  manpower  shortage,  their  long- 
term usefulness  is  limited. 

Updating  of  Data  Base;  It  is  important  to  recognize  that 
the  geographic  and  transit  data  bases,  once  established, 
are  not  static.  Rather,  they  must  be  constantly  changed 
to  reflect  changes  in  local  land  use  and  in  transit  service. 
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Making  these  changes  requires  a full-time  effort  of  one 
WMATA  employee.  Data  base  updates  are  made  so  frequently 
that  the  AIDS  geographic  data  base  is  becoming  recognized 
as  the  most  accurate  address  directory  available  for  the 
D.C.  metropolitan  area.  Anticipated  cost  savings  due  to 
centralized  data  base  updating,  however,  have  not  been 
achieved  due  to  the  fact  that  hard-copy  reference  data  are 
still  being  made  available  to  all  agents  as  a backup  to 
AIDS. 

Agent  Job  Satisfaction:  All  data  collected  about  AIDS' 

influence  on  job  satisfaction  point  to  the  fact  that  AIDS 
was  well-received  by  agents.  In  the  before-implementation 
survey,  73%  of  agents  responding  chose  "excellent"  or 
"good"  to  describe  their  job  satisfaction;  this  percentage 
rose  following  implementation  to  85%.  On  the  after- 
implementation questionnaire,  agents  were  asked  to  rate 
AIDS'  influence  on  their  job  satisfaction.  To  this  question, 
76%  replied  "positive  influence"  and  the  remainder  replied 
"no  influence".  None  of  the  responding  agents  responded 
"negative  influence". 

Data  Collection;  The  AIDS  software  as  implemented  contains 
a number  of  features  which  enhance  system  usefulness  as  a 
data  collection  tool.  These  features  are  currently  being 
routinely  used  by  supervisory  and  management  personnel  to 
monitor  individual  agent  as  well  as  system  performance. 

Since  implementation,  there  has  been  a dramatic  increase 
in  both  the  quantity  and  quality  of  these  performance 
data. 

Interdepartmental/Spinoff  Use;  AIDS  implementation  has 
raised  potentials  for  interdepartmental  cooperation  within 
WMATA  on  a scale  not  heretofore  realized.  Examples  of 
cooperative  efforts  that  have  already  been  realized  are  as 
follows : 
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The  WMATA  Scheduling  Department  has  benefited  from 
standardization  and  storage  of  its  data  on  word  pro- 
cessing equipment.  While  this  standardization  and 
storage  effort  was  a necessary  expenditure  for  the 
establishment  of  the  AIDS  transit  data  base,  it  now 
costs  the  Scheduling  Department  less  to  update 
schedules.  Since  this  Scheduling  process  is  ongoing, 
these  savings  will  continue  indefinitely. 

The  AIDS  transit  data  base  was  used  as  a resource  by 
the  WMATA  Planning  Department  in  the  recent  imple- 
mentation of  the  automated  RUCUS  driver  scheduling 
and  run-cutting  system.  One  marketing  official 
estimated  that  having  this  transit  data  fully 
organized  saved  WMATA  over  $50,000  in  implementation 
costs  for  RUCUS. 

The  WMATA  Planning  Department  has  also  made  use  of 
the  AIDS  Daily  Log  File  as  a planning  resource.  For 
example,'  origins  and  destinations  of  all  itinerary 
calls  can  be  superimposed  against  a system  route  map 
to  determine  whether  service  is  adequate  in  areas 
with  high  information  demand.  Frequency  of  demand 
for  information  on  certain  bus  routes  can  also  be 
compared  with  frequency  of  service  on  these  routes. 
Examples  of  potential  benefits  yet  to  be  realized  are  as 
follows : 

The  WMATA  Bus  Stop  Department  can  make  use  of  AIDS  in 
its  ongoing  program  of  providing  information  at 
MetroBus  Stops.  AIDS  data  can  be  used  to  provide 
printed  information  as  to  which  bus  routes  pass  by  a 
given  stop,  for  display  at  the  stop. 

The  WMATA  Accounting  and  Finance  Department  can  use 
the  AIDS  transit  data  base  as  a resource  in  calcu- 
lating the  number  of  passenger  route-miles  provided 
by  WMATA  to  each  of  the  various  local  jurisdictions 
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in  the  service  area.  Using  the  data  base  in  this 
fashion  would  greatly  simplify  these  calculations, 
which  form  one  basis  for  WMATA's  operating  subsidy 
assessments.  Given  alternative  assumptions  about 
route  and  service  level  changes,  these  data  can  also 
be  used  as  a financial  forecasting  tool,  to  show  the 
cost  effects  of  proposal  service  changes. 

Additionally,  AIDS  implementation  has  also  raised  the 
potential  for  other  non-WMATA  agencies  to  make  use  of 
the  system.  Examples  of  such  "spinoff"  possibilities 
are  the  following:  First,  use  of  the  geographic  data 

base  by  the  Federal  Government  or  other  jurisdictions 
as  a resource  in  ridesharing-car  pool  programs. 

Second,  provision  of  the  geographic  base  as  a planning 
resource  to  local  governments.  (It  is  generally 
recognized  that  this  data  base  is  the  most  accurate 
street  address  directory  of  the  D.C.  metropolitan 
area  in  existence.  The  D.C.  Council  of  Governments 
has  already  expressed  a willingness  to  pay  for  use  of 
this  geographical  data  base.)  Finally,  provision  of 
remote  AIDS  terminals  at  such  local  transportation 
centers  for  use  by  the  general  public.  (Such  centers 
might  include  the  larger  METRO  stations.  National  and 
Dulles  Airports,  local  universities,  etc.) 

Thus,  the  implementation  of  AIDS  and  the  concomitant  esta- 
blishment of  its  various  data  bases  have  created  a significant 
resource  which  has  strong  potential  for  serving  a number 
of  users  besides  information  operators  and  callers.  The 
overall  financial  justification  of  the  system  is  therefore 
far  greater  than  merely  productivity  increases  and  cost 
savings  within  the  WMATA  Office  of  Marketing.  According 
to  the  AIDS  Program  Manager,  the  benefits  created  by  AIDS 
have  already  greatly  outweighed  its  overall  cost  to  the 
Authority.  These  benefits  include  not  only  the  substantial 
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monetary  gain  to  WMATA  due  to  a 10-15%  increase  in  operator 
productivity.  They  also  include  less  tangible  factors 
such  as  increased  accuracy  and  consistency  of  information 
provided,  enhanced  operator  job  satisfaction,  and  use  of 
the  unified  transit  data  base  supporting  AIDS  as  a resource 
by  other  WMATA  departments  and  local  government  agencies. 

In  conclusion,  it  is  evident  that  the  system  implementation 
has  enabled  the  WMATA  Office  of  Marketing  to  achieve  (at  least 
partially)  all  of  the  benefits  described  above.  With  the  full 
cooperation  of  management  and  users,  future  system  enhancements 
should  increase  the  overall  success  of  AIDS. 
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1.0  INTRODUCTION 


This  report  presents  a description  of  the  data  collection 
and  analysis  efforts  undertaken  in  the  evaluation  of  the 
Automated  Information  Directory  System  (AIDS) , an  automated 
transit  information  system  (ATIS)  recently  developed  and 
implemented  by  the  Office  of  Marketing  at  the  Washington,  D.C., 
Metropolitan  Area  Transit  Authority.  The  WMATA  ATIS  deployment 
is  the  most  ambitious  and  comprehensive  example  to  date  of  a 
family  of  automated  information  systems  whose  principal 
objective  is  to  reduce  costs,  increase  productivity  and  improve 
service  to  the  transit-riding  public.  These  systems  have  been 
funded  from  research  and  development  through  implementatiori 
over  the  past  eight  years  by  the  Urban  Mass  Transportation 
Administration's  Office  of  Methods  and  Support.  The  objectives 
of  this  report  are  to  quantify  and  assess  the  impacts  of  this 
application  of  ATIS  technology  on  system  users  and  management 
within  WMATA,  as  well  as  on  the  general  public  in  the 
Washington,  D.C.,  area  being  served  by  the  new  system.  It  is 
hoped  that  the  descriptions  of  system  implementation  efforts  in 
this  report  will  also  serve  to  provide  helpful  insights  for 
other  transit  properties  interested  in  the  potential  application 
of  ATIS  technology  to  their  own  public  information/marketing 
functions. 


2.0  AUTOMATION  OF  TRANSIT  INFORMATION  SERVICES 

2.1  THE  ROLE  OF  TELEPHONE  INFORMATION  IN  TRANSIT  MARKETING 
Transit  properties  have  at  their  disposal  a variety  of 
media  for  marketing  their  services  to  the  general  public. 
Among  these  are  printed  schedules  and  maps,  radio,  television 
and  newspaper  advertising,  and  telephone  information  dissemi- 
nated from  a central  office.  As  compared  with  other  media,  a 
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telephone  information  service  has  several  distinct  advantages. 
First,  it  is  readily  accessible  to  a substantial  portion  of  the 
transit-riding  public.  Second,  it  can  be  easily  updated  to 
reflect  changes  in  service.  Third,  and  most  important,  it  can 
be  personalized  to  suit  the  needs  of  the  individual  customer. 

For  those  customers  who  do  not  possess  printed  transit  schedules 
or  maps,  calling  the  transit  telephone  information  service  is  a 
convenient  method  of  learning  how  to  make  a particular  trip  via 
transit.  Even  those  customers  who  possess  printed  transit 
schedules  or  maps  often  find  it  useful  to  call  such  a service 
in  order  to  interpret  or  verify  information.  For  these  reasons, 
particularly  in  urban  transit  properties  having  complex  systems 
in  terms  of  size,  number  of  routes,  or  alternative  modes,  a 
telephone  information  service  is  generally  employed  to  overcome 
a major  barrier  to  transit  use:  public  apprehension  of  "getting 
lost"  on  transit. 

2.2  TRADITIONAL  TRANSIT  INFORMATION  CENTER  CHARACTERISTICS 

In  a typical  transit  system  telephone  information  center, 
system  employees  (who  will  be  referred  to  as  "agents")  answer 
inquiries  from  prospective  transit  riders  about  schedules, 
routes,  particular  trip  itineraries,  fares,  etc.  Traditionally, 
the  agent  receives  the  caller's  inquiry,  and,  if  necessary, 
assists  the  caller  in  stating  it  as  exactly  as  possible.  Once 
the  inquiry  is  understood,  the  agent  consults  route  maps, 
schedules,  headway  sheets,  or  other  indexed  printed  information 
located  at  the  agent's  work  station,  piecing  together  the 
information  desired.  Once  an  appropriate  response  has  been 
retrieved,  it  is  provided  to  the  caller. 

Under  this  traditional  manual  system  of  data  retrieval, 
the  agent's  command  of  interpersonal  communications  skills  is 
particularly  important.  Of  equal  importance  under  this  system 
is  the  agent's  personal  knowledge  of  both  transit  system  and 
local  geographical  characteristics.  In  order  to  be  able  to 
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locate  quickly  a desired  trip's  origin  and  destination  and  to 
select  effectively  appropriate  transit  routings  between  them, 
the  agent  must  undergo  intensive  training  as  well  as  on-the-job 
skill  development  before  full  competency  is  achieved.  Training 
and  skill  development  is  especially  necessary  in  properties 
where  the  transit  system  is  complex  and/or  its  service  area  is 
large. 

2.3  THE  POTENTIAL  FOR  AUTOMATION  OF  AGENT  DATA  RETRIEVAL 

FUNCTIONS 

Advances  in  computer  technology  over  the  past  decade  have 
raised  the  potential  for  automating  data  retrieval  functions 
performed  by  transit  information  agents.  Certain  aspects  of 
these  functions  lend  themselves  well  to  computer  applications. 
First,  they  are  highly  repetitive;  a very  large  percentage  of 
caller  inquiries  can  be  broken  down  into  only  four  or  five 
distinct  call  types,  each  requiring  consultation  of  certain 
specific  data  types.  Thus,  programming  the  computer  for  data 
retrieval  is  a relatively  manageable  task  of  duplicating  agent 
behavior  elicited  by  a relatively  limited  number  of  distinct 
call  types.  Second,  these  data  retrieval  functions  require  a 
high  degree  of  agent  familiarity  with  service  area  geography 
and  transit  system  characteristics.  These  characteristics  can 
be  digitized  on  a coordinate  grid  system  and  mapped  into  the 
memory  of  a computer.  Third,  agent  functions  often  involve 
optimization  of  response  data  to  conform  to  limitations  placed 
on  certain  trip  parameters  by  the  caller.  For  example,  the 
caller  might  want  to  minimize  total  travel  time,  the  number  of 
transfers  required,  walking  distances  from  stop  to  destination, 
etc.  The  logic  underlying  agent  response  selection  in  view  of 
these  caller  stated  limitations  can  be  translated  into 
comparison/optimizing  computer  routines  in  a very  straightforward 
manner.  Fourth,  it  is  very  important  that  agent  data  retrieval 
exhibit  a high  degree  of  consistency.  While  human  error  or 
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varying  agent  preferences  or  work  habits  may  introduce  a 
certain  amount  of  variability  in  agent  responses,  machine  logic 
ensures  that  the  same  inquiry  will  elicit  the  same  computer 
response  every  time.  Finally,  due  to  high  public  demand  for 
their  services,  agents  must  work  as  quickly  and  efficiently  as 
possible  in  retrieving  data.  The  highspeed  calculating  capa- 
bilities of  a computer  are  thus  particularly  appropriate  in 
this  application.  Except  for  a relatively  small  percentage  of 
cases  in  which  agents  are  able  to  answer  inquiries  "off  the  top 
of  their  heads"  or  very  rapidly  look  up  data,  automated 
retrieval  is  faster  than  manual  consultation  of  reference 
materials.  For  detailed  inquiries  involving  multiple-transfer 
trips,  it  is  substantially  (minutes)  faster. 

Thus,  automation  of  data  retrieval  functions  appears  to  be 
a technically  feasible  means  of  improving  a telephone  infor- 
mation agent's  capabilities  in  terms  of  response  accuracy, 
consistency,  and  speed.  The  agent's  job,  however,  is  different 
under  an  automated  transit  information  system.  ATIS  operation 
involves  an  agent's  receiving  telephone  queries  from  a caller 
and  retrieving  desired  information  through  use  of  a remote 
interactive  computer  terminal  located  at  the  agent's  work 
position.  The  system  is  activated  when  query  data  are  entered 
on  the  terminal  keyboard  by  the  agent.  These  data  are  then 
processed  in  the  computer  through  pre-programmed  routines 
selected  by  the  agent  which  reference  the  geographical  and 
transit  data  bases  (addresses,  fares,  routes,  schedules,  etc.) 
and  optimize  (e.g.  minimize)  certain  caller-stated  trip  para- 
meters (travel  time,  transfers,  fare,  walking  distances,  etc.) 
through  comparison  of  potentially  feasible  routing  alternatives. 
Following  processing,  which  is  accomplished  in  a matter  of 
seconds,  the  most  appropriate  responses  considering  given  caller- 
stated  limitations  are  displayed  on  a cathode  ray  tube  (CRT) 
located  above  the  agent's  keyboard.  This  display  is  then  evaluated 
by  the  agent  and  the  appropriate  information  supplied  to  the 
caller . 
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2.4  POTENTIAL  ATIS  IMPLEMENTATION  BENEFITS 

In  light  of  the  above  discussion,  it  appears  that  a well- 
designed  and  implemented  automation  of  a transit  information 
center  would  produce  benefits  to  both  the  transit  property  and 
the  general  public.  These  benefits  include  the  following: 

1.  Improved  information  agent  service  and  productivity 
resulting  from  faster  data  retrieval  by  the  computer. 

2.  Increased  reliability  and  consistency  of  agent 
responses,  since  the  computer  should  always  supply 
the  same  answer  to  the  same  question. 

3.  Enhanced  agent  satisfaction  and  improved  attitudes 
deriving  from  the  elimination  of  some  of  the  more 
tedious  aspects  of  their  jobs. 

4.  A reduction  in  training  time  for  information  agents, 
since  extensive  training  in  area  geography  and  transit 
system  characteristics  is  no  longer  necessary  - this 
information  would  be  stored  in  the  computer's,  rather 
than  the  agent's,  memory. 

5.  Improved  capability  to  rapidly  incorporate  schedule 

or  other  service  changes  into  the  information  center's 
reference  data  base,  since  such  changes  would  require 
only  a simple  modification  of  the  computer's  memory 
accessible  to  all  agents,  rather  than  a reprinting  or 
modification  of  hard  copy  reference  data  for  each. 

6.  Expanded  capability  to  gather  statistics  about  tele- 
phone information  center  operations  since  the  computer 
can  be  programmed  to  store  and  compile  records  on  all 
agent-directed  functions  and  processing  translations. 

7.  The  geographical  and  transit  system  data  bases  used 

as  an  integral  part  of  the  ATIS  could  prove  a valuable 
source  of  information  for  local  government  agencies 
or  other  transit  system  departments  such  as  those 
dealing  with  route  planning,  stop  locations,  and 
service  schedules. 
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3.0  ATIS  IMPLEMENTATION  AT  WMATA 


3.1  WMATA  INFORMATION  SERVICE  CHARACTERISTICS 

The  telephone  information  facility  in  VJMATA's  Office  of 
Marketing  possesses  a number  of  characteristics  that  made  it  a 
prime  candidate  for  application  of  ATIS  technology.  WMATA’ s 
transit  system  is  multi-modal,  complex,  and  growing;  its  service 
area  covers  1500  square  miles  and  16  separate  political  juris- 
dictions, extending  out  from  the  District  of  Columbia  into 
Virginia  and  Maryland  suburbs.  Within  this  area,  WMATA  operates 
buses  on  over  800  routes  in  200  separate  transit  corridors,  as 
well  as  a rapid  rail  system  that  is  projected  to  include  87 
stations  and  100  miles  of  rail  when  completed  in  the  late  1980 's. 
As  of  1982,  39.12  miles  of  this  rail  system  were  in  revenue 
service  and  an  additional  33.52  miles  were  under  construction.^ 

On  an  average  day  the  WMATA  telephone  information  facility 
receives  between  five  and  ten  thousand  calls  on  fifty  different 
trunk  lines.  To  meet  this  demand,  WMATA  employs  a total  of  57 
agents  and  7 supervisory  personnel  who  answer  calls  in  3 shifts, 
18  hours  a day,  365  days  a year.  Despite  this  considerable 
investment  in  manpower,  between  30  and  40  percent  of  all  incoming 
calls  are  "lost"  during  peak  demand  periods;  that  figure  repre- 
sents those  callers  placed  on  hold  by  the  telephone  equipment 
because  all  agents  are  busy,  and  who  subsequently  hang  up  before 
reaching  an  agent  or  receiving  any  information. 


1.  Jeremy  F.  Plant,  The  Metrorail  System;  Its  Impact  On 
Virginia  (Charlottesville,  VA:  University  of  Virginia 
Institute  of  Government  Newsletter,  September  1982),  P.2. 
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3.2  AIDS  PROJECT  ORIGIN  AND  BACKGROUND  (1972-1974) 

The  origins  of  the  AIDS  project  can  be  traced  to  the  early 
1970 's  when  research  carried  out  in  several  locations  pointed 
to  potential  cost-effective  applications  of  computer  technology 
to  data  retrieval  functions.  Research  undertaken  at  the  National 
Bureau  of  Standards  (NBS)  and  at  the  MITRE  Corporation  in  the 
Washington  area  investigated  the  costs  and  benefits  of  automated 
data  retrieval,  and  work  on  route-finding  algorithms  conducted 
at  the  System  Development  Corporation  (SDC)  in  Santa  Monica, 
California,  culminated  in  the  development  of  the  Passenger 
Routing  System  (PARIS) , a proprietary  system  which  was  success- 
fully demonstrated  by  SDC  at  the  Santa  Monica  transit  system  in 
1974.  While  the  Santa  Monica  transit  system  was  not  very  complex, 
this  demonstration  proved  the  system's  capabilities  for  pro- 
viding responses  within  a competitive  time-frame;  that  is,  at 
speeds  equal  to  or  better  than  manual  agent  retrieval  speeds. 

The  problem  of  providing  accurate  or  reliable  telephone  infor- 
mation to  potential  transit  customers  was  one  which  had  plagued 
the  private  bus  companies  who  were  the  predecessors  of  WMATA. 

When  IVMATA  was  created  by  interstate  compact  in  1973,  these 
services  were  merged  and  placed  under  central  management. 
Determined  to  improve  this  service,  and  having  been  convinced 
by  the  earlier  MITRE  and  NBS  studies  that  computerization  held 
considerable  promise,  the  Director  of  the  newly-created  Office 
of  Marketing  at  WMATA  was  able  to  obtain  a Federal  matching 
capital  grant  for  2/3  of  the  $675,000  estimated  as  necessary  to 
fund  installation  of  a computer  information  system.  The 
non-Federal  portion  of  this  grant  was  included  as  a line  item 
in  WMATA's  first  capital  budget. 

3.3  FEASIBILITY  STUDY  (1974-1975) 

With  this  grant  secured,  WMATA  proceeded  with  a feasibility 
study  for  a computer  information  system.  In  a competitive 
process,  the  feasibility  study  contract  was  awarded  to  Systems 
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Consultants,  Inc.  (SCI) , of  Washington.  This  contract  included 
investigation  of  input  data  requirements,  system  features, 
possible  hardware  configurations,  and  system  layout  requirements. 

As  the  study  progressed,  various  technical  and  managerial 
problems  were  encountered  which  eventually  resulted  in  WMATA's 
having  work  stopped  under  the  contract.  Thus,  in  1975,  having 
spent  a considerable  amount  of  the  $675,000,  the  Office  of 
Marketing  was  left  with  several  disorganized  feasibility  study 
products,  including  a number  of  preliminary  system  design  tasks 
outside  the  scope  of  the  original  study. 

3.4  SYSTEM  SPECIFICATION  (1976-1977) 

It  was  at  this  point  that  WMATA  officials  recognized  the 
potential  for  Federal  assistance  in  designing  and  implementing 
the  AIDS  project  through  the  Technology  Development  and 
Demonstration  Program.  Aware  that  what  remained  of  the 
original  $675,000  capital  grant  would  only  fund  the  hardware 
and  "capital"  costs  of  the  demonstration,  UMTA  officials  approved 
an  additional  100%  Federal  R&D  grant  in  the  amount  of  $435,000 
to  complete  the  preliminary  work.  This  grant  was  used  by  WMATA 
to  assimilate  the  SCI  study  products,  research  other  necessary 
requirements,  synthesize  them  into  a comprehensive  specification 
for  the  computerized  transit  information  retrieval  system,  and 
develop  the  software  needed  to  operate  the  system.  Since  such 
a system  had  never  been  implemented  on  such  a large  data  base 
as  would  be  required  by  WMATA,  these  were  felt  to  be  legitimate 
research  costs  for  such  a system. 

The  remciining  analysis  tasks  and  the  system  specification 
were  accomplished  in  a cooperative  effort  between  the  Offices 
of  Marketing  and  Data  Processing  at  WMATA  and  the  METREK 
Division  of  the  MITRE  Corporation  in  McLean,  Virginia,  where 
some  of  the  initial  work  investigating  data  retrieval  systems 
had  been  accomplished.  This  effort  took  over  a year,  and  the 
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Review  and  Specifications  for  AIDS  was  published  in  July  1977.* 

It  was  during  the  course  of  this  study  that  many  major 
decisions  were  made  that  had  a profound  effect  upon  ultimate 
system  design.  One  such  decision  was  to  incorporate  the  entire 
WMATA  service  area  in  the  system  all  at  once,  rather  than 
beginning  with  a small  demonstration  and  phasing  in  additional 
sectors  as  implementation  progressed.  This  decision  was  based 
on  the  confidence  of  1#1ATA  in  system  integrity  and  on  the 
availability  of  funds  to  do  the  whole  job.  Another  such  decision 
was  the  incorporation  of  dedicated  computer  hardware  rather 
than  relying  on  WMATA' s Data  Processing  Office.  This  decision, 
coupled  with  budget  considerations,  dictated  that  a mini-computer 
configuration  be  employed,  rather  than  a large  mainframe  computer. 
As  a result,  a number  of  limitations  on  data  base  size  and 
system  software  were  introduced.  The  primary  reason  behind 
this  choice  of  system  configuration  was  the  desire  for  indepen- 
dence and  control  over  computing  functions  on  the  part  of  the 
Marketing  Office.  In  a rapid  time  responsive  environment. 

Office  of  Marketing  officials  felt  that  they  could  not  afford 
to  be  held  up  by  decisions  on  the  part  of  the  separate  Office 
of  Data  Processing  regarding  scheduling  of  AIDS  processing 
vis-a-vis  other  time-consuming  (and  response-slowing)  data 
processing  jobs,  such  as  payroll  computation  and  generation. 

3.5  CONTRACTOR  SELECTION  (1977) 

Based  upon  the  Review  and  Specifications,  the  Office  of 
Marketing  at  WMATA  issued  a Formal  Request  for  Proposals  for 
system  design  in  late  1977.  Although  the  RFP  generated  con- 
siderable interest  (about  280  were  sent  out  worldwide) , only 


* M.P.  Munkasey,  M.L.  Noonchester,  and  D.F.  O'Sullivan,  An 
Automated  Information  Directory  System  (AIDS)  - Review  and 
Specifications  PB  272  253  UMTA  Report  No.  VA-06-0038-77-1 , July 
1977. 
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five  proposals  were  received.  One  of  these  five  responses  was 
judged  to  be  a superficial  effort,  and  was  eliminated  from 
consideration  quite  early  in  the  review  process. 

The  remaining  four  system  design  proposals  were  judged 
separately,  first  with  regard  to  technical  merit  and  then  with 
regard  to  cost.  As  negotiations  with  each  of  the  four  bidders 
progressed,  both  the  bidders  and  the  Contract  Evaluation  Board 
which  was  assembled  to  review  the  bids,  modified  their  original 
assumptions  and  expectations  in  a process  of  mutual  education. 
Major  technical  evaluation  criteria  included: 

1.  Understanding  of  large  data  base  systems 

2.  Experience  with  geographic  data  bases 

3.  Various  skills  represented  on  the  contractor 
management  team 

4.  Experience  with  route-finding  algorithms 

In  order  that  each  of  the  four  bidders  be  given  the  opportunity 
to  better  define  its  proposal  in  light  of  information  obtained 
in  the  negotiation  process,  WMATA  issued  a "Request  for  Best 
and  Final  Offer"  for  which  responses  were  due  December  1,  1977. 
Based  upon  the  responses  to  this  request,  the  Contract 
Evaluation  Board  chose  Kappa  Systems,  Inc.  as  system  design 
contractor.  This  choice  was  based  on  several  factors: 

• overall  understanding  of  the  technical  problems 
involved  in  system  design, 

• experience  with  satellite  tracking  systems  which  used 
an  algorithm  similar  to  that  for  AIDS,  and 

• willingness  to  work  with  WMATA  management  and  users 
in  refining  the  final  product. 

3.6  SYSTEM  IMPLEMENTATION  (1978-1981) 

Kappa  and  WMATA  officially  began  the  AIDS  implementation 
process  in  March  1978.  Initial  system  implementation  work  was 
focused  on  three  separate  tasks  which  proceeded  concurrently: 

• WMATA  began  a series  of  major  design  changes  to  the 
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layout  of  its  Office  of  Marketing  in  the  WMATA 
Operations  Control  Building  in  order  to  accommodate 
the  computer  system.  These  changes  included  instal- 
lation of  new  operator  work  stations,  supervisory 
offices,  a training  room,  and  a computer  room.  They 
also  included  installation  of  heavy  heating,  ventila- 
ting, and  air  conditioning  (HVAC)  equipment  for 
precise  climate  control  in  the  computer  room  as  well 
as  ventilation  of  the  operator  work  areas.  Since  the 
telephone  information  center  operates  long  past  normal 
working  hours,  it  was  considered  desirable  to  control 
its  ventilation  independently  of  the  HVAC  equipment 
needed  to  service  the  rest  of  the  building.  These 
improvements  cost  the  Authority  approximately  $250,000. 
Kappa  began  the  task  of  generating  the  geographic 
data  base,  which  included  all  the  street  addresses  in 
the  WMATA  service  area  as  well  as  over  1000  landmarks. 
All  these  addresses  and  landmarks  were  digitized  on  a 
coordinate  grid.  In  order  to  obtain  resolution  down 
to  the  individual  street  address,  it  was  necessary  to 
construct  an  extremely  detailed  grid,  with  individual 
elements  approximately  25  feet  on  a side.  Thus,  in  a 
single  square  mile,  there  were  44,000  geographic 
elements;  in  the  entire  service  area  there  were  well 
over  50,000,000  such  elements!  While  digitized  census 
data  was  used  in  some  cases,  it  proved  to  be  unreliable 
for  the  entire  service  area;  many  sections  of  the 
geographic  data  base  required  slow,  painstaking, 
labor-intensive  hand-digitizing  with  little  opportunity 
for  double-checking;  even  the  telephone  directory 
proved  to  be  of  little  value.  In  the  digitizing 
effort,  several  recurring  problems  slowed  progress. 

For  example,  it  was  necessary  to  arbitrarily  define, 
in  many  instances,  the  point  at  which  the  "north"  and 
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"south"  ends  of  a particular  street  commenced.  Rotary 
traffic  intersections,  which  occur  quite  frequently 
in  Washington,  posed  particular  problems  of  inter- 
section definition. 

• WMATA  began  the  task  of  assembling  transit  data  base 
information,  which  was  to  be  provided  to  Kappa  in 
machine-readable  form.  From  the  outset,  it  became 
apparent  that  this  task  was  difficult;  the  infor- 
mation was  generated  and  updated  by  several  different 
WMATA  departments  each  with  its  own  system  for  infor- 
mation storage  and  formatting. 

Faced  with  the  problem  of  how  to  standardize  the  poorly 
formatted  Scheduling  Department  data,  which  were  produced  on 
antiquated  machines  and  stored  as  hard  copy  (often  modified 
with  tape  and  white-out) , the  Office  of  Marketing  successfully 
convinced  WMATA' s General  Manager  that  significant  cost  savings 
in  updating  could  be  realized  if  the  schedule  data  were  totally 
reformatted  on  new  word  processing  equipment  and  stored  on 
floppy  disks.  In  order  to  effect  this  change,  the  Office  of 
Marketing  had  to  purchase  the  word  processing  equipment  and 
train  Scheduling  Department  personnel  in  its  use.  The  revised 
schedule  data  were  organized  in  a standardized  format,  using 
time  reference  points  that  were  logical  from  an  operational 
point  of  view. 

A similar  problem  was  confronted  in  obtaining  usable  bus 
stop  location  information.  Many  transit  authorities  of  com- 
parable size  have  entire  departments  dedicated  to  the 
compilation  of  bus  stop  data;  WMATA  at  this  point  had  only  one 
employee  dedicated  to  this  effort,  and  information  storage  was 
again  on  hard  copy.  Both  Office  of  Marketing  and  Kappa  personnel 
were  assigned  to  assist  this  person  in  putting  together  a computer 
directory  of  all  bus  stops.  A number  of  problems  hindered  this 
effort.  First,  the  assignment  of  addresses  to  bus  stops  was 
rather  arbitrary,  but  these  addresses  had  to  be  clearly  defined 
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due  to  the  fine  grid  of  the  data  base.  Second,  a number  of 
separate  bus  lines  utilized  the  same  stops.  Stops  had  to  be 
clearly  assigned  to  each  bus  route  they  served  for  data  base 
purposes.  This  problem  had  not  presented  itself  under  the 
previous  manual  system  of  bus  stop  management.  Solving  this 
assignment  problem  was  quite  a labor-intensive  task.  Third,  the 
sheer  number  of  WMATA  bus  stops  (over  11,000)  made  the  data 
validation  process  even  more  cumbersome.  Finally,  the  fact 
that  stop  locations  were  changing  throughout  the  process  added 
to  the  complexity  of  the  problem. 

In  light  of  the  above  discussion  it  is  evident  that  both 
WMATA  and  Kappa  underestimated  the  complexity  of  the  initial 
three  system  start-up  tasks  (construction/HVAC , geographic  and 
transit  data  base  definition) . Delays  were  encountered  in  all 
three  areas  which  slowed  the  implementation  process  considerably. 
A further  exacerbating  factor  was  the  unexpected  departure  to 
another  firm  of  the  Kappa  Project  Manager  and  several  key  Kappa 
staff  employees.  An  interim  manager  was  named,  but  little  else 
was  accomplished.  Because  WMATA  was  still  experiencing  con- 
struction delays  and  transit  data  base  difficulties  during  this 
period,  however.  Kappa  had  some  excuse  for  non-performance. 
Eventually,  shortly  after  construction  of  the  facility  was 
finished,  WMATA  marketing  officials  began  to  pressure  Kappa  to 
fulfill  their  contractual  obligations.  At  this  point  a third 
Kappa  project  manager  was  named,  and  under  his  direction  the 
long  process  of  system  data  entry  was  accomplished  in  late 
1980.  The  total  cost  to  the  Authority  for  the  data  base  was 
approximately  $275,000. 

3.7  SYSTEM  ACCEPTANCE  AND  AGENT  TRAINING 

Although  Information  Center  supervisory  personnel  had 
taken  an  extremely  active  role  throughout  the  implementation 
process  (the  Senior  Supervisor  even  sat  on  the  original  Contract 
Evaluation  Board) , the  agents  themselves  were  only  marginally 
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involved  until  the  data  validation  and  system  checkout  process 
began  in  1980.  Agent  curiosity  concerning  the  system,  however, 
was  quite  high.  When  Kappa  set  up  a demonstration  terminal  for 
agents  to  experiment  with,  all  but  four  of  the  75  center 
employees  tried  it  out  at  one  time  or  another. 

Agents  played  a crucial  role  in  the  system  acceptance 
process,  however.  In  essence,  they  "fine-tuned"  the  system 
with  their  extensive  knowledge  of  WMATA  service  characteristics. 
Kappa  personnel  produced  long  lists  of  computer  responses  to 
certain  routing  questions  and  alternative  response  screen 
formats  for  agents  to  mark  up,  stating  preferences  or  reasons 
for  observed  "errors".  It  was  through  this  feedback  process 
that  Kappa  determined  the  system  of  weight  assignments  which 
dictated  why  one  routing  alternative  might  be  assigned  higher 
priority  than  another  and  the  final  response  screen  format  for 
each  of  the  four  basic  system  functions  (itinerary,  schedule, 
route,  fare).  The  attitude  on  the  part  of  Kappa  of  "we  need 
your  help"  was  well  received  by  the  operators,  who  prided  them- 
selves on  this  extensive  knowledge.  System  design  personnel 
and  WMATA  management  alike  felt  that  a process  of  open  dialogue 
was  crucial  to  the  usefulness  of  the  system,  as  agent  judgment 
in  making  choices  between  alternatives  was  the  ultimate  source 
of  information  in  making  these  refinements.  During  this  process, 
agents  were  enthusiastic  participants,  actively  volunteering 
their  time  to  ensure  the  best  end  product  possible. 

It  was  during  this  period  that  the  first  agent  training 
sessions  were  held.  Before  implementation  of  the  ATIS,  the 
standard  agent  training  course  at  WMATA  lasted  approximately 
six  weeks,  during  which  the  trainees  received  intensive  drilling 
on  geographical,  route  and  service  characteristics,  reference 
materials  and  their  usage,  and  interpersonal  communications 
skills.  To  achieve  full  proficiency  on  the  system  required  an 
additional  "apprenticeship"  period  of  approximately  six  months 
of  full-time  work. 
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Although  training  agents  to  use  AIDS  had  originally  been 
Kappa's  responsibility  under  the  terms  of  the  system  develop- 
ment contract r WMATA  management  had  been  sensitized  by  some 
difficulties  during  initial  contractor  agent  training  for  the 
other  UMTA-sponsored  ATIS  demonstration  at  the  Southern 
California  Rapid  Transit  District  (SCRTD)  in  Los  Angeles.  At 
the  SCRTD  the  contractor  training  efforts  appeared  to  have  led, 
for  various  reasons,  to  operator  mistrust  of  the  system.  Not 
desirous  of  repeating  this  pattern  of  events,  WMATA  officials 
assumed  responsibility  for  agent  training.  They  took  a training 
manual  for  agents  which  had  been  produced  by  Kappa  and  reworked 
it  to  focus  on  aspects  of  the  system  which  were  particularly 
useful  from  the  agent's  perspective.  The  first  two  training 
classes,  consisting  of  four  agents  each,  were  carefully  selected 
by  management  for  their  overall  maturity  and  experience,  their 
receptiveness  to  changes  in  routine,  and  for  their  capabilities, 
once  trained,  to  set  an  enthusiastic  example  for  the  other 
agents.  Agent  training  consisted  of  typing  practice  (development 
of  typing  ability  was  considered  important  since  many  agents 
lacked  this  skill)  and  practice  manipulating  the  various  functions 
(itinerary,  schedule,  route  and  fare)  as  well  as  instruction  in 
how  to  quickly  scan  alternative  responses  (the  system  usually 
produces  four  alternative  responses  t.o  an  operator  query)  to 
select  the  "best"  response  in  light  of  the  caller's  request. 

Agents  in  the  early  training  sessions  were  instructed  to  keep 
any  negative  comments  about  the  system  to  themselves,  at  least 
for  the  short  run.  It  was  generally  conceded  by  Kappa  that  the 
geographic  and  transit  data  bases  contained  numerous  errors; 
agents  were  openly  told  that  these  errors  might  occur,  and  that 
part  of  their  job  was  to  assist  Kappa  in  finding  and  correcting 
them.  This  process  was  greatly  facilitated  by  a built-in  error 
reporting  function  in  the  system  software.  If  an  agent  received 
a response  which  he  or  she  deemed  questionable,  it  could  easily 
be  stored  for  future  reference  and  correction.  Sessions  in 
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which  these  questionable  responses  were  discussed  by  agents  and 
Kappa  representatives  proved  to  be  particularly  fruitful  and 
instructive  to  both  parties.  In  many  instances,  the  minimum- 
travel  time  routing  selected  by  the  computer  was  not  the 
"intuitive"  or  "obvious"  choice  selected  by  the  agent.  For 
example,  the  computer  might  instruct  a caller  to  walk  several 
blocks  and  catch  an  express  bus  rather  than  catch  a local  bus 
for  which  no  v^alking  was  required.  Agents  in  many  such  instances 
had  to  reluctantly  concede  that  the  computer's  selection  was 
indeed  better  from  t.he  standpoint  of  travel  time.  In  some 
instances,  however,  agent  judgment  was  better.  For  example,  a 
certain  computer  routing  might  at  first  glance  seem  to  be  the 
best  choice,  but  it  might  contain  a segment  which  the  agent 
considered  inadvisable  or  impossible.  Typical  agent  comments 
to  these  routings  were  "I  wouldn't  transfer  in  that  neighborhood 
at  that  time  of  night.",  or  "How  can  you  cross  that  highway  to 
reach  the  transfer  stop?",  etc.  In  these  instances,  the  agent's 
detailed  knowledge  of  local  pedestrian  traffic  routings  out- 
weighed the  pre-programmed  computer  response.  While  there  were 
initially  a considerable  number  of  error  reports,  these  gradually 
tapered  off  as  corrections  were  made  and  system  acceptance  was 
fully  complete  in  February  1981. 

It  should  be  emphasized  that  agent  training  at  this  stage 
of  AIDS  development  was  limited  to  experienced  agents  who  had 
already  received  the  intensive  six-week  training  course  for  new 
agents.  As  of  the  writing  of  this  report,  WMATA  experience  in 
training  new  agents  directly  on  AIDS  was  extremely  limited. 

4.0  SYSTEM  EVALUATION 

With  the  full  cooperation  of  the  management  and  staff  of 
WMATA's  Office  of  Marketing,  Wilson  Hill  Associates  in  the 
Spring  of  1980  designed  a comprehensive  program  for  collection 
and  evaluation  of  data  concerning  AIDS.  These  data  were  collected 
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in  April-May,  1980,  November,  1980,  and  Oc tober-November-December , 
1981.  The  major  goals  of  this  program  were  to  determine,  as 
accurately  as  possible,  answers  to  the  following  questions: 

• In  what  ways  does  AIDS  affect  the  Telephone  Information 
Section's  productivity,  i.e.,  the  overall  quantity  of 
agent  responses  provided? 

• In  what  ways  does  AIDS  affect  the  quality,  (i.e.,  the 
accuracy  and  consistency)  of  agent  responses  provided? 

• How  does  AIDS  affect  an  average  agent's  performance 
of  normal  job  functions?  How  does  it  change  the 
agent's  perception  of  his/her  job? 

• What  impacts  does  AIDS  have  on  the  training  process 
for  new  agents? 

• What  other  impacts,  positive  or  negative,  does  the 
system  exert  on  other  WMATA  departments  and  on  the 
general  public? 

A number  of  different  data  collection  instruments  were 
implemented  as  elements  of  this  evaluation  program,  taking 
advantage  of  a variety  of  different  information  sources. 

Sources  employed  in  obtaining  both  quantitative  and  qualitative 
data  are  described  below. 

4.1  QUANTITATIVE  DATA  SOURCES 

In  evaluating  the  impacts  of  AIDS  on  a quantitative  basis, 
there  were  two  primary  objectives.  The  first  was  to  measure 
the  characteristics  of  caller  demands  placed  on  the  information 
center.  The  second  was  to  gain  a better  understanding  of  those 
aspects  of  operator  performance  which  could  be  quantified. 

Several  different  sources  were  employed  in  this  effort: 

• Automatic  Call  Distribution  System  readings 

• Call  monitoring 

• Computer  Data 
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The  Automatic  Call  Distribution  (ACD)  system,  manufactured 
by  Stromberg-Carlson , routes  incoming  calls  to  the  information 
agents.  When  a call  is  received,  it  is  either  routed  to  an 
available  agent  or,  if  all  agents  are  busy,  it  is  routed  to  a 
holding  queue.  Once  a call  is  placed  on  this  queue,  it  receives 
a recorded  message  stating  that  all  operators  are  busy  and 
instructing  callers  in  preparing  their  inquiries.  Calls  are 
routed  from  the  queue  on  a first-in,  first-out  basis;  that  is, 
the  caller  waiting  "on  hold"  for  the  longest  time  is  answered 
first. 

The  ACD  system  proved  to  be  a valuable  source  of  data 
concerning  both  caller  demand  and  agent  performance.  Using  a 
device  called  a "peg  counter",  it  mechanically  records  the 
number  of  calls  entering  the  system  and  the  number  of  calls 
which  are  "lost",  i.e.,  those  which  are  ended  by  the  caller 
before  reaching  an  agent.  On  a standard  form,  incoming  calls 
and  lost  calls  are  recorded  from  the  counter  each  hour  by  WMATA 
supervisory  personnel.  Once  these  data  are  recorded,  the  ACD 
counter  is  reset  to  zero  for  the  next  hour's  count.  ACD  data 
is  used  extensively  by  WMATA  as  an  indication  of  agent  and 
center  performance  as  well  as  caller  demand  levels. 

Call  monitoring  is  another  technique  used  routinely  by 
WMATA  supervisory  personnel  to  check  on  agent  performance.  All 
agent  work  positions  are  fitted  with  jacks  so  that  additional 
headsets  can  be  plugged  in  for  monitoring.  Call  monitoring 
proved  useful  in  obtaining  more  detailed  information  concerning 
individual  operator  performance  than  was  available  from  the  ACD 
system  counts.  For  example,  call  content,  agent  data  retrieval 
time,  data  sources  referenced,  and  the  total  call  time  were 
recorded  in  this  manner.  Of  course,  since  monitoring  was  limited 
in  focus  to  an  individual  agent,  it  was  necessary  to  monitor  a 
large  number  of  calls  handled  by  many  different  agents  in  order 
to  gain  statistical  confidence  regarding  monitoring  results. 
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The  AIDS  system,  once  operational,  proved  to  be  a valuable 
source  of  agent  performance  data  as  well.  System  specifications 
for  AIDS  required  a "report  generator"  program  be  included  in 
the  system  capable  of  "printing  information  handled  through  the 
central  computer".*  A series  of  summary  reports  of  daily  computer 
activity  are  printed.  Typical  data  included  in  these  reports 
are : 

By  Hour  of  the  Day: 

• Maximum  number  of  active  AIDS  terminals 

• Number  of  itinerary,  schedule,  route  and  fare  queries 
completed 

• Number  of  "errors" 

• Number  of  "keyed  functions" 

By  Individual  Agent; 

• Agent  name  and  ID  number 

• Number  of  minutes  logged-in 

• Number  of  itinerary,  schedule,  route  and  fere  "replies 
received" 

• Number  of  "errors" 

In  these  reports  "errors"  include  any  inputs  which  are  not 
recognizable  elements  of  the  computer's  data  base.  They  may  be 
the  result  of  incorrect  caller  information,  agent  misspellings, 
or  other  data  entry  errors.  "Keyed  functions"  are  all  data 
inputs,  including  errors.  "Replies  received"  are  useful  data 
outputs  which  do  not  include  system  error  messages. 


* Munkasey,  Noonchester  and  O'Sullivan,  op.  cit. , p.  9. 
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4.2  QUALITATIVE  DATA  SOURCES 

The  major  objectives  of  the  qualitative  data  collection 
effort  were  to  gain  useful  information  regarding  those  aspects 
of  AIDS  operation  that  were  difficult  or  impossible  to  quantify. 
Such  information  -included  agent  attitudes  towards  AIDS  and 
their  opinions  regarding  its  accuracy,  its  impact  on  their  job 
functions,  its  effects  on  callers,  its  influence  on  their  job 
satisfaction,  etc.  It  also  included  WMATA  management  perceptions 
as  to  how  system  implementation  was  progressing,  how  effectively 
it  was  being  used  by  the  agents,  etc.  Two  techniques  were  used 
to  gather  these  data: 

• Survey  questionnaires  of  operators 

• Direct  interviews  of  WMATA  personnel  involved  in 
system  implementation  as  well  as  information  agents. 

Questions  asked  in  the  agent  opinion/attitude  question- 
naires were  generally  multiple-choice  in  format  although  several 
questions  also  provided  blanks  for  agents  to  supply  additional 
written  responses  if  they  desired  to  do  so.  Overall  length  of 
the  questionnaires  was  limited  to  approximately  15  questions  to 
ensure  maximum  participation  on  the  part  of  the  agents.  Responses 
were  coded  numerically  and  processed  to  yield  frequencies  of 
each  possible  response,  as  well  as  cross-tabulation  of  responses. 

Interviews  of  management  personnel  and  operators  attempted 
to  fccus  on  their  perceptions  of  AIDS'  impacts  on  their  jobs 
and  the  jobs  of  their  subordinates.  These  impact  areas  were 
often  discussed  by  the  interviewees  more  in  terms  of  future  or 
expected  results  than  actual  verified  impacts.  This  focus  was 
due  to  the  relative  newness  of  the  system  and  the  fact  that 
agents  were  still  adjusting  to  it.  Interviews  were  very  helpful, 
however,  in  revealing  agent  job  styles  and  in  determining  costs 
and  benefits  of  the  system. 

4.3  EVALUATION  METHODOLOGY 

The  methodology  employed  by  Wilson  Hill  in  evaluating  AIDS 
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was  the  product  of  two  major  determining  factors: 

• The  institutional  support  for  full  AIDS  imple- 
mentation on  the  part  of  WMATA  management;  and 

• The  timing  of  system  implementation. 

WMATA  Office  of  Marketing  officials  had  few  misgivings  concerning 
the  potential  effectiveness  of  AIDS.  This  attitude  was  reflected 
in  their  decisions  to  incorporate  full  service-area  coverage 
and  dedicated  hardware  into  the  system  and  to  train  all  operators 
in  its  use.  It  was  not  considered  important  by  these  officials, 
therefore,  to  experimentally  validate  the  usefulness  of  com- 
puterized data  retrieval  against  the  manual  mode.  Rather,  the 
major  question  on  the  part  of  WMATA  was:  "How  well  are  the 
agents  making  use  of  the  computer  system?"  Most  evaluation 
components  were  designed  with  this  question  in  mind. 

The  timing  of  system  implementation  was  particularly 
fortuitous  for  evaluation  purposes.  When  the  evaluation  began. 
Kappa  was  still  putting  AIDS  through  its  system  acceptance 
testing  and  the  system  was  not  yet  operational.  The  operators 
in  the  Telephone  Information  Section  were  therefore  still  using 
manual  data  retrieval  to  handle  all  calls.  This  circumstance 
presented  the  opportunity  for  collection  of  data  under  the 
manual  mode  which  could  later  be  used  as  a baseline  for  com- 
pai ison  with  data  to  be  gathered  when  AIDS  was  fully  opera- 
tional and  agents  were  trained  and  familiar  with  its  use.  This 
"before-and-af ter " approach  was  used  for  evaluation  of  ACD , 
call  monitoring,  and  agent  survey  data. 

4.4  BEFORE-IMPLEMENTATION  ACD  DATA 

In  the  initial  data  collection  phase,  it  was  considered 
useful  to  examine  a limited  amount  of  Automatic  Call  Distri- 
bution System  data  to  gain  an  overview  of  caller  demand 
magnitudes  and  the  percentage  of  calls  answered  and  "lost" 
under  manual  operation.  The  figures  shown  in  Table  4-1  are 
typical  of  the  ACD  data.  They  depict  total  calls  received. 
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TABLE  4-1 

TYPICAL  "BEFORE"  ACD  DATA 


DATE 

(1980) 

' TOTAL 
CALLS 

CALLS 

ANSWERED 

CALLS 

LOST 

PERCENT 

ANSWERED 

4/20 

5823 

5278 

545 

90.6 

4/21 

8069 

6854 

1215 

84.9 

4/22 

7115 

6669 

446 

93.7 

4/23 

7009 

6503 

506 

92.8 

4/24 

6711 

6479 

232 

96.5 

4/25 

6486 

6321 

165 

97.5 

4/26 

5755 

4780 

975 

83.1 

4/27 

5021 

4478 

543 

89.2 

4/28 

8762 

6678 

2084 

76.2 

4/29 

7650 

7425 

225 

97.1 

4/30 

6842 

6559 

283 

95.9 

5/1 

6729 

6368 

361 

94.6 

5/2 

6877 

6407 

470 

93.2 

5/3 

6529 

5463 

1066 

83.7 

AVERAGE 

6813 

6162 

651 

90.6 

STANDARD 

DEVIATION 

905 

807 

508 
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calls  answered,  calls  "lost"  and  percentages  answered  over  a 
two  week  period  from  April  20  - May  3,  1980.  During  this  period, 
daily  calls  received  ranged  from  5021  to  8762,  with  an  average 
of  6813  and  a standard  deviation  of  905.  Daily  calls  answered 
ranged  from  4478  to  7425,  with  an  average  of  6162  and  a standard 
deviation  of  807.  The  most  interesting  feature  of  these  ACD 
statistics  is  their  fluctuation  throughout  the  week.  Figure 
4-1  plots  total  calls  and  answered  calls  as  a total  of  average 
weekly  calls  received.  In  this  figure,  it  can  be  seen  that 
caller  demand  peaks  at  approximately  17.5%  of  weekly  demand  on 
Mondays,  and  declines  steadily  throughout  the  week  to  a level 
slightly  below  13%  on  Saturdays  and  Sundays.  This  peaking  of 
caller  demand  on  Mondays  is  also  reflected  in  the  "percent 
answered"  figures  in  the  last  column  of  Table  4-1.  From  these 
figures,  it  can  be  seen  that  the  agent  work  force  was  insufficient 
to  meet  Monday  demands  (4/21  and  4/28)  but  quite  adequate  on 
the  other  weekdays.  Calls  answered  range  between  13  and  14.5% 
of  weekly  demand  throughout  the  week  and  decline  to  slightly 
above  10%  on  weekends.  This  weekend  variation  can  be  explained 
by  the  smaller  agent  work  force  available  to  answer  calls  on 
weekends. 

Thus,  the  ACD  data  gathered  before  system  implementation 
indicates  that  the  center  did  a very  good  job  of  handling  demand 
on  Tuesdays  through  Fridays,  with  an  average  of  95%  of  calls 
answered  on  these  days.  On  Saturday,  Sunday  and  Monday,  center 
staffing  was  evidently  inadequate  to  handle  demand,  and  an 
average  of  only  85%  of  calls  were  answered  on  these  days. 

4.5  BEFORE- IMPLEMENTATION  CALL  MONITORING 

In  order  to  gain  as  much  information  as  possible  concerning 
agent  work  habits  and  job  styles  under  manual  operation,  a call 
sampling  effort  was  undertaken  over  a period  of  six  weeks,  from 
October  20  to  November  26,  1980.  In  this  effort,  incoming 
calls  were  selected  at  random  by  a part-time  Wilson  Hill  employee 
located  at  a supervisory  work  station  in  the  Customer  Information 
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PERCENT  OF  AVERAGE  WEEKLY  CALLS  RECEIVED 


FIGURE  4-1 

DAILY  VARIATION  IN  CALLS  RECEIVED  AND  CALLS  ANSWERED 

BEFORE  IMPLEMENTATION 
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Source:  ACD  Records 
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section.  From  this  location,  operators  could  be  unobtrusively 
monitored  and  observed.  For  each  call  monitored,  a separate 
line  of  data  was  recorded  by  the  observer  on  a data  sheet. 

Data  recorded  included  time  of  day,  operator  monitored,  total 
elapsed  time  of  call,  type  of  query,  materials  consulted  by 
operator,  mode  of  transit  described  in  operator  response,  and 
any  other  qualitative  descriptions  or  call  "problems"  that  were 
considered  relevant  (for  example,  a caller  comprehension  problem 
which  resulted  in  the  operator's  repeating  instructions).  In 
all,  a total  of  1,789  calls  were  successfully  documented  in 
this  fashion.  Raw  data  sheets  were  then  coded  numerically, 
transcribed  into  a computer  file,  and  processed  using  a Statistical 
Package  for  the  Social  Sciences  (SPSS)  computer  program.  Output 
of  this  program  consisted  of  frequencies  for  each  variable 
recorded  as  well  as  a variety  of  two-dimensional  cross- tabulations. 

The  large  amount  of  data  collected  in  this  before- implementa- 
tion sampling  effort  permits  generally  reliable  statements  to 
be  made  concerning  the  call  characteristics  recorded.  Findings 
regarding  these  characteristics  are  described  below. 

• An  overwhelming  majority  of  the  calls  monitored  involved 
relatively  direct  and  straightforward  requests  for  infor- 
mation. In  a total  of  1,258,  or  71.4%  of  these  calls, 
only  one  or  two  specific  caller  requests  were  made  (i.e. 
concerning  a specific  itinerary  or  schedule) . An  additional 
411,  or  23.3%,  involved  three  or  four  requests.  Only  in 

93  of  the  calls,  or  5.3%  of  the  total,  were  there  five  or 
more  distinct  caller  queries.  This  tendency  towards  brief 
caller  requests  is  obviously  a contributing  factor  to  high 
operator  call  productivity  in  the  Customer  Information 
Section. 

• A total  of  1,256,  or  76.1%  of  those  calls  in  which  a transit 
mode  was  mentioned,  dealt  exclusively  with  bus  transportation. 
135  calls,  or  8.2%,  requested  information  about  rail  transit 
only.  In  260  calls,  or  15.7%,  both  modes  were  mentioned. 

One  possible  explanation  for  this  observed  distribution  is 
the  fact  that  only  a portion  of  the  planned  WMATA  rail 
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system  was  in  operation,  while  the  bus  network  is  quite 
extensive  and  complex.  Another  explanation  might  be  that 
marketing  promotions  of  the  rail  transit  system  have  made 
it  much  more  familiar  to  area  residents  than  the  WMATA 
Metrobus  system. 

Information  was  also  recorded  regarding  the  content  of 
caller  queries.  In  many  cases,  a certain  amount  of 
judgment  on  the  part  of  the  person  monitoring  the  calls 
was  required  for  this  task.  Customer  queries  were  classi- 
fied as  itinerary,  schedule,  route,  fare,  or  other  requests. 
The  "other"  category  included  a variety  of  queries,  the 
majority  of  which  were  related  to  senior  citizen  discounts 
or  special  flash-pass  programs.  Itinerary  requests  were 
by  far  the  most  popular,  accounting  for  two-thirds  of  the 
total.  Schedule  requests  were  the  next  most  frequent, 
comprising  24%.  Fare  requests  made  up  7%  of  the  total 
while  route  and  other  requests  accounted  for  the  remaining 
3%. 

Because  the  person  monitoring  the  calls  could  observe 
telephone  operators  in  action,  it  was  possible  to  record 
the  types  of  reference  materials  consulted  in  retrieving 
data  for  callers.  Four  separate  categories  of  reference 
material  were  recorded:  maps  only,  printed  materials  only, 
both  maps  and  printed  materials,  or  none  (agent  memory  of 
system) . The  frequency  with  which  agents  can  answer 
queries  "off  the  top  of  their  heads"  without  consulting 
any  reference  has  strong  implications  for  overall  system 
productivity,  since  such  responses  involve  no  data 
retrieval  time  and  are  generally  completed  quickly.  In 
the  1,305  instances  in  which  reference  use  was  recorded, 
agents  consulted  only  maps  on  367  calls,  or  28.1%  of  the 
total;  they  used  only  printed  materials  on  535  calls,  or 
41%  of  the  total;  and  both  maps  and  schedules  on  212  calls, 
or  16.2%  of  the  total.  They  were  able  to  answer  191,  or 
14.6%  of  all  queries,  "off  the  top  of  their  heads". 
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Total  elapsed  time  of  each  call  monitored  was  recorded 
using  a stopwatch.  The  vast  majority  of  calls  were 
relatively  brief;  a total  of  751,  or  42.1%  of  the  1,782 
calls  whose  times  were  recorded,  lasted  one  minute  or 
less.  An  additional  565,  or  31.7%  of  the  total,  lasted 
between  one  and  two  minutes.  There  were  234  calls  with 
elapsed  times  between  two  and  three  minutes,  13.7%  of  the 
total.  Ninety-seven  calls,  or  5.4%  of  the  total,  lasted 
between  three  and  four  minutes.  Only  135  calls,  or  7.6% 
of  the  total,  took  longer  than  four  minutes  to  complete. 
This  call  distribution  reflects  an  average  of  slightly 
more  than  two  minutes,  or  an  average  operator  productivity 
of  approximately  25  to  30  calls  per  hour,  assuming  the 
operator  works  continuously  without  any  breaks  between 
calls.  Figure  4-2  plots  the  frequency  distribution 
described  above. 

Included  in  the  data  sheet  used  by  the  person  monitoring 
calls  was  a section  in  which  notations  could  be  made 
concerning  any  problems  which  caused  longer  than  usual 
call  times.  In  coding  the  data  sheets,  these  notations 
were  classified  into  four  separate  categories:  calls  in 
which  the  operator  repeated  information  to  the  caller, 
calls  in  which  the  caller  could  not  understand  the 
operator's  response,  calls  in  which  the  caller  was 
unfamiliar  with  locations  described  by  the  operator, 
and  calls  in  which  the  operator  was  unable  to  find 
locations  referred  to  by  the  caller.  No  problems  were 
noted  by  the  observer  in  1,391  calls,  or  77.8%  of  the 
total.  Agents  repeated  information  on  251  calls,  or 
14%  of  the  total.  Customer  comprehension  problems  were 
noted  on  25  calls,  or  1.4%  of  the  total.  In  86  calls, 
or  4.8%  of  the  total,  the  caller  did  not  recognize 
locations  described  by  the  operator.  In  only  36  instances, 
or  2%  of  the  total,  was  the  operator  unable  to  find 
locations  described  by  the  customer. 
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PERCENT  OF  ALL  CALLS  MONITORED 


FIGURE  4-2 

FREQUENCY  DISTRIBUTION  OF  CALL  TIMES 
BEFORE  IMPLEMENTATION  CALL  MONITORING 


CALL  DURATION  (MINUTES) 
Source:  WHA  Call  Monitoring 
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All  but  one  of  the  cross-tabulations  produced  from  the 
coded  call  monitoring  data  incorporated  a time  interval 
(hour,  day,  week)  as  one  of  the  variables.  Because 
data  were  not  collected  continuously,  and  only  a small 
percentage  of  total  incoming  calls  were  monitored,  it 
is  not  possible  to  make  broad  generalizations  from  the 
distributions  over  time  of  the  variables  as  recorded. 

For  example,  the  fact  that  call  totals  for  a particular 
hour  or  day  of  the  week  were  higher  than  those  for 
other  hours  or  days  does  not  necessarily  imply  that  the 
customer  information  section  was  busier  during  these 
periods;  it  merely  means  that  the  person  collecting  the 
data  was  busier.  Table  4-2  presents  the  single  cross- 
tabulation generated  in  data  processing  which  does  not 
have  a unit  of  time  as  one  of  the  variables.  In  this 
table,  the  frequency  of  use  of  reference  materials  is 
tabulated  against  the  distribution  of  call  times.  Exami 
nation  of  this  table  yields  some  interesting  conclusions 
First,  those  calls  in  which  operators  consulted  only 
maps  generally  took  longer  to  complete  than  those  in 
which  only  printed  materials  were  referenced.  Approxi- 
mately 25%  of  the  "maps  only"  calls  were  completed  in 
less  than  one  minute,  while  slightly  under  half  the 
"printed  materials  only"  calls  were  completed  in  this 
time.  Second,  those  calls  which  required  an  agent  to 
consult  both  reference  types  had  generally  the  longest 
durations;  38%  of  these  calls  had  durations  of  over 
three  minutes.  Finally,  as  might  be  expected,  those 
calls  in  which  no  reference  materials  are  used  generally 
took  the  shortest  time  to  complete;  approximately  two- 
thirds  of  these  calls  lasted  less  than  one  minute. 
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TABLE  4-2 

CROSS-TABULATION  OF  REFERENCE  USE  VS  CALL  TIME 
BEFORE  IMPLEMENTATION  CALL  MONITORING 


REFERENCES 

CONSULTED 

CALL  TIME  (MINUTES) 

TOTAL 

0-1 

1-2 

2-3 

3-4 

OVER  4 

MAPS  ONLY 

95 

151 

69 

24 

28 

367 

PRINTED  MATERIALS 
ONLY 

262 

175 

55 

15 

26 

533 

MAPS  AND  PRINTED 
MATERIALS 

13 

66 

51 

29 

51 

210 

NO  REFERENCES 

127 

48 

14 

1 

1 

191 

TOTAL 

497 

441 

189 

69 

106 

1302 

PERCENT  OF 
TOTAL 

38.2 

33.9 

14.5 

5.3 

8.1 

Source:  WHA  Call  Monitoring 
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4.6  BE FORE -IMPLEMENTATION  AGENT  QUESTIONNAIRE 

The  before- implementation  agent  questionnaire  was 
designed  and  administered  to  WMATA  operators  during  November 
1980.  This  survey  was  designed  to  encourage  maximum  partici- 
pation on  the  part  of  the  agents.  Overall  length  was  limited 
to  14  questions,  most  of  which  were  presented  in  a multiple-choice 
format.  On  five  questions,  however,  a blank  was  provided 
for  "open-ended"  responses  by  agents  who  chose  to  volunteer 
additional  information.  Agent  response  to  the  survey  was 
excellent;  a total  of  42  surveys  were  completed  and  returned 
for  processing.  Responses  were  coded  numerically  and  analyzed 
using  a Statistical  Package  for  the  Social  Sciences  (SPSS) 
computer  program.  This  processing  yielded  frequencies  for 
each  possible  response  as  presented  below. 

• To  the  question  "How  long  have  you  worked  as  a Transit 
Information  Agent?",  three  agents  responsed  "six  months 
or  less",  eight  responded  "seven  to  twelve  months";  13 
responded  "one  to  three  years";  nine  responded  "three 
to  five  years";  and  nine  responded  "more  than  five 
years".  Average  agent  experience  reflected  by  this 
distribution  is  at  least  25  months.  This  distribution 
is  shown  in  the  last  column  of  Table  4-4. 

• The  number  of  calls  handled  by  an  agent  in  an  hour  is 
the  standard  for  productivity  at  WMATA.  V^hen  asked 
"How  many  calls  do  you  process  per  hour  under  busy 
conditions?",  two  agents  replied  "15  or  below";  five 
replied  "16  to  20";  15  replied  "21  to  25";  12  replied 
"25  to  30";  and  eight  replied  over  30".  This  response 
distribution  represents  an  average  agent  productivity 
of  at  least  22  calls  per  hour.  It  is  shown  in  the 
bottom  row  of  Table  4-4. 


31 


When  asked  "Do  you  feel  your  skill  at  the  job  increases 
with  time?"  24  agents  responded  "yes";  and  17  agents 
responded  "does  for  a while  and  then  levels  off".  None 
of  the  agents  surveyed  chose  the  "no"  response.  This 
result  would- indicate  strong  support  for  intensive  agent 
training  during  the  first  few  months  of  call  processing 
to  increase  agent  competence  and  familiarity  with  the 
system. 

Agents  were  asked  to  rate  the  physical  requirements  of 
their  job  on  the  average  day.  Three  separate  response 
categories  were  provided,  along  with  a blank  labeled 
"explain"  for  agents  who  cared  to  supply  any  more  infor- 
mation on  the  subject.  Eleven  agents  rated  the  job  as 
"very  fatiguing";  27  termed  it  "somewhat  fatiguing";  and 
only  three  agents  chose  the  "little  or  no  fatigue"  response. 
A total  of  28  agents  added  explanations  of  these  responses 
in  the  blank  provided.  Of  these,  six  responses  might  be 
classified  under  the  general  complaint  of  "tedious  and 
repetitive";  seven  mentioned  having  to  "sit  all  day",  and 
15  noted  "eye  strain",  "mental  strain",  or  "headaches" 
caused  by  the  job. 

A series  of  five  questions  dealt  with  the  reference 
materials  available  to  the  agents  under  the  existing 
manual  system  of  data  retrieval.  These  materials  include 
system  maps,  route  descriptions,  and  headway  sheets. 

Also  included  under  "reference  materials"  was  the  category 
"personal  knowledge  of  the  system"  since  agents  are  some- 
times able  to  respond  to  caller  requests  without  consulting 
any  printed  reference  material.  When  asked  which  of 
these  materials  they  considered  the  most  valuable,  18 
agents  replied  "maps";  10  replied  "personal  knowledge  of 
the  system";  five  replied  "route  descriptions";  five 
replied  "headway  sheets";  and  one  replied  all  of  the 
above.  This  distribution  is  shown  in  the  bottom  row  of 
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Table  4-5. 

Table  4-3  shows  results  obtained  when  agents  were 
asked  how  frequently  they  consulted  the  references  mentioned 
above.  As  can  be  seen  in  this  table,  headway  sheets  are 
the  most  frequently  used,  followed  by  personal  knowledge, 
maps  and  route  descriptions,  respectively.  The  frequency 
with  which  agents  are  able  to  respond  to  caller  queries 
from  memory  is  an  important  factor  in  overall  productivity, 
since  this  type  of  response  requires  no  data  retrieval 
time.  The  fact  that  this  is  the  second  most  popular 
"reference"  consulted  is  reflected  in  the  high  agent 
productivity  figures  reported  above. 

Agents  were  given  a choice  of  five  alternative  responses 
to  the  question  "What,  in  your  opinion,  is  the  bigqest 
problem  in  processing  calls?",  plus  a blank  in  which 
additional  information  could  be  supplied.  A total  of  18 
out  of  40  agents  who  answered  this  questions  chose  the 
response  "customer  understanding  of  information";  10 
agents  chose  "keeping  up  with  changes  in  routes,  schedules 
and  fares";  four  chose  "understanding  customer's  question"; 
three  chose  "customer  impatience";  and  two  chose  "retrieving 
information  in  reference  material".  Eight  agents  provided 
additional  information  in  the  blank.  Of  these,  five 
emphasized  that  getting  customer  cooperation  was  their 
biggest  problem,  while  three  stated  all  of  the  factors 
listed  were  equally  important. 

Agents  responded  generally  favorably  to  the  question  "How 
satisfied  are  you  with  your  job?"  Seven  agents  replied 
"very  satisfied";  and  24  replied  "somewhat  satisfied", 
while  10  replied  "unsatisfied"  and  only  one  replied  "very 
unsatisfied" . 

For  the  question  "What  duties  do  you  like  most  about  your 
job?",  no  alternative  answers  were  provided;  just  a blank 
for  agents  to  supply  their  own  comments.  A total  of  35 
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TABLE  4-3 

AGENT-REPORTED  FREQUENCY  OF  REFERENCE  USE 
BEFORE  IMPLEMENTATION  AGENT  SURVEY 


REFERENCE 

MATERIAL 

FREQUENCY  OF  USE 

HARDLY 
AT  ALL 

SOMETIMES 

FREQUENTLY 

VERY 

FREQUENTLY 

PERSONAL 

KNOWLEDGE 

1 

8 

24 

9 

MAPS 

1 

16 

16 

9 

ROUTE 

DESCRIPTIONS 

12 

20 

10 

0 

HEADWAY 

SHEETS 

2 

2 

14 

24 

TOTAL 

16 

46 

64 

42 

% OF  ALL 
RESPONSES 

10 

27 

38 

25 
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TABLE  4-4 

CROSS -TABULATION  OF  AGENT  EXPERIENCE  VS  CALL  COUNT 
BEFORE  IMPLEMENTATION  AGENT  SURVEY 


EXPERIENCE 
AS  AN  AGENT 

CALL  COUNT  PER  HOUR 

TOTAL 

15 

OR  LESS 

16- 

20 

21- 

25 

26- 

30 

OVER 

30 

6 MO.  OR  LESS 

1 

2 

0 

0 

0 

3 

7-12  MOS. 

1 

2 

3 

1 

1 

8 

1-3  YRS. 

0 

0 

6 

5 

2 

13 

3-5  YRS. 

0 

1 

3 

4 

1 

9 

OVER  5 YRS. 

0 

0 

3 

2 

4 

9 

TOTAL 

2 

5 

15 

12 

8 

42 

35 


agents  responded.  The  most  frequent  response,  mentioned 
by  a total  of  17  agents,  might  generally  be  termed  "helping 
people".  Other  responses  included  "working  with  the 
public",  "helping  customers  save  money"  and  "giving  good, 
accurate  information". 

To  the  question  "What  duties  do  you  dislike  most  about 
your  job?",  once  again  only  a blank  was  provided  for 
agent  comment.  A total  of  35  agents  responded  with  a 
wide  variety  of  comments,  including  "management  pressure 
for  higher  call  counts",  "repetition"  or  "monotony", 

"dealing  with  difficult  customers",  "callers  who  use  foul 
language"  and  "poor  quality  of  reference  materials". 

The  final  questions  on  the  survey  dealt  with  agent  per- 
ceptions of  changes  to  their  jobs  due  to  the  implemen- 
tation of  AIDS.  While  none  of  the  agents  had  yet  received 
training  on  AIDS,  they  were  all  aware  that  its  installation 
was  being  planned;  they  had  seen  demonstrations  of  its 
use.  To  the  question,  "What  effect  do  you  think  the  new 
computerized  information  system  (AIDS)  will  have  on  your 
job?",  two  agents  chose  the  response  "no  change";  24 
agents  chose  "some  improvement";  10  chose  "big  improvement"; 
and  five  chose  "be  worse".  This  distribution  is  shown  in 
Table  4-6.  A blank  was  provided  in  which  agents  could 
explain  their  response  if  they  so  chose;  a total  of  31 
agents  provided  comments.  Those  agents  who  felt  AIDS 
would  be  an  improvement  generally  mentioned  retrieval 
speed  and  the  accuracy  and  consistency  of  the  data  it 
would  provide.  Some  agents  stated  that  it  would  make 
their  jobs  "more  professional".  Those  who  felt  it  would 
have  no  effect  or  would  make  their  job  worse  for  the  most 
part  made  comments  such  as  "computers  are  too  slow", 
"information  will  be  inaccurate",  or  "I'll  have  to  wait 
and  see". 

A number  of  two-dimensional  cross-tabulat ions  were  run  on 
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the  response  data  collected  in  an  attempt  to  identify  any 
interesting  patterns  that  might  not  have  been  evident  in  the 
response  frequencies.  Tables  4-4  through  4-7  present  four 
such  cross-tabulations. 

• In  Table  4-4  agent  experience  responses  are  tabulated 
against  their  responses  indicating  agent  call  count  per 
hour.  Looking  at  this  table,  it  can  be  seen  that  there 
is  a frequency  of  responses  along  the  diagonal  running 
from  the  upper  left  corner  to  the  lower  right  corner. 

This  pattern  lends  support  to  the  notion  that  agent  call 
counts  increase  with  experience,  a result  that  was  obtained 
in  the  third  survey  question  ("Do  you  feel  your  skill  at 
the  job  increases  with  time?"),  to  which  agent  responses 
were  strongly  affirmative. 

• In  Table  4-5  agent  call  count  responses  are  tabulated 
against  their  responses  concerning  reference  materials 
considered  most  valuable.  From  this  table,  it  can  be 
seen  that  eight  of  those  agents  with  call  counts  greater 
than  25  per  hour  stated  that  they  considered  personal 
knowledge  of  the  system  to  be  more  important  than  any 
other  reference  type.  Conversely,  only  two  agents  with 
call  counts  below  25  per  hour  cited  personal  knowledge  as 
the  most  important  reference.  This  result  would  lend 
credence  to  the  hypothesis  that  increased  productivity  is 
strongly  dependent  on  the  frequency  with  which  agents  can 
respond  to  caller  queries  "off  the  top  of  their  heads". 

• Table  4-6  presents  responses  concerning  agent  experience 
tabulated  against  agent  responses  concerning  feelings 
about  AIDS  system  implementation.  From  this  table,  it 
can  be  seen  that  the  generally  favorable  agent  disposition 
towards  AIDS  changeover  is  reflected  across  all  experience 
categories.  Even  those  agents  with  over  five  years' 
experience  using  manual  retrieval  felt  quite  strongly 
(seven  agents  to  two)  that  AIDS  would  cause  at  least  some 
improvement  in  their  jobs. 
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TABLE  4-5 


CROSS-TABULATION  OF  CALL  COUNT  VS  REFERENCE  MATERIALS 
CONSIDERED  MOST  IMPORTANT 

BEFORE  IMPLEMENTATION  AGENT  SURVEY 


CALL  COUNT 
PER  HOUR 

REFERENCE  MATERIALS 
CONSIDERED  MOST  IMPORTANT 

TOTAL 

PERSONAL 

KNOWLEDGE 

SYSTEM 

MAPS 

ROUTE 

DESCRIPTION 

HEADWAY 

SHEETS 

OTHER 

15  Or  Less 

0 

1 

1 

0 

0 

2 

16  - 20 

1 

2 

0 

1 

1 

5 

21  - 25 

1 

5 

4 

2 

3 

15 

26  - 30 

4 

7 

0 

1 

0 

12 

Over  30 

4 

3 

0 

1 

0 

8 ■ 

TOTAL 

10 

18 

5 

5 

4 

42 
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TABLE  4-6 


CROSS -TABULATION  OF  AGENT  EXPERIENCE  VS  FEELINGS 
CONCERNING  AIDS  IMPLEMENTATION 

BEFORE  IMPLEMENTATION  AGENT  SURVEY 


EXPERIENCE 
AS  AN  AGENT 

FEELINGS  CONCERNING  AIDS 

TOTAL 

NO 

CHANGE 

SOME 

IMPROVEMENT 

BIG 

IMPROVEMENT 

BE 

WORSE 

0 - 6 MO. 

0 

3 

0 

0 

3 

7 - 12  MO. 

0 

6 

0 

1 

7 

1-3  YRS. 

0 

7 

4 

2 

13 

3-5  YRS. 

1 

4 

3 

1 

9 

OVER  5 YRS. 

1 

4 

3 

1 

9 

TOTAL 

2 

24 

10 

5 

41 
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• Table  4-7  cross- tabulates  the  same  responses  concerning 

AIDS  changeover  shown  in  Table  4-6  versus  explanations  of 
these  feelings  provided  in  the  open-ended  "please  explain" 
following  the  question.  From  this  table,  it  can  be  seen 
that  the  four  agents  who  think  AIDS  implementation  will 
have  a detrimental  effect  on  their  jobs  stated  that  they 
would  take  a "wait  and  see"  attitude  until  the  system  was 
implemented.  Those  agents  who  felt  AIDS  would  improve 
their  job  generally  cited  the  faster  retrieval  speed  and 
improved  accuracy  and  consistency  of  the  AIDS  data  base 
as  compared  with  the  manual  references  currently  available. 

4.7  AFTER- IMPLEMENTATION  ACD  DATA 

In  the  post- implementation  data  collection  phase,  ACD 
data  recorded  by  supervisory  personnel  in  the  Telephone  Infor- 
mation Section  were  once  again  made  available  to  Wilson  Hill. 

These  data  are  indicative  of  section  performance  under  steady-state 
operation  of  AIDS. 

It  should  be  noted  that  AIDS  was  implemented  as  a supple- 
ment to  manual  retrieval  and  that  during  the  post- implementa- 
tion data  collection  phase  all  agents  still  had  access  to  and 
training  in  the  use  of  manual  data  sources.  Thus,  even  in 
full  operation,  not  all  calls  resulted  in  AIDS'  transactions 
being  processed.  Thus,  the  AIDS  daily  log  file,  while  pre- 
senting a very  accurate  description  of  system  transactions, 
did  not  provide  a total  picture  of  post- implementation  Telephone 
Information  Section  operations.  For  this  comprehensive  picture, 
the  ACD  data  were  still  required. 

Table  4-8  presents  a summary  of  the  af ter-impleirentation 
ACD  data,  which  cover  the  period  from  October  19  to  November 
30,  1981.  This  is  a longer  period  than  the  pre- implementation 
data  made  available;  results  can  therefore  be  viewed  as 
slightly  more  reliable  than  those  for  pre- implementation. 

During  this  period  total  daily  calls  received  ranged  from  4116 
to  9232,  with  an  average  of  5985  and  a standard  deviation  of 
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TABLE  4-7 


CROSS -TABULATION  OF  FEELINGS  CONCERNING 
AIDS  IMPLEMENTATION  VS  EXPLANATION  OF  THESE  FEELINGS 

BEFORE  IMPLEMENTATION  AGENT  SURVEY 


AGENT  FEELINGS 
REGARDING  AIDS 

EXPLANATION 

OF  THESE 

FEELINGS 

FASTER 

RE- 

SPONSE 

IMPROVED 

INFOR- 

MATION 

PRESENT 

SYSTEM 

BETTER 

WAIT 
AND  SEE 

OTHER 

NO  CHANGE 

0 

0 

0 

0 

2 

SOME 

IMPROVEMENT 

8 

2 

1 

0 

1 

BIG 

IMPROVEMENT 

6 

3 

0 

0 

0 

BE  WORSE 

0 

0 

0 

4 

0 

41 


TABLE  4-8 

AFTER- IMPLEMENTATION  ACD  DATA 
(AVERAGES  FOR  6-WEEK  DATA  COLLECTION  PERIOD) 


DAY 

OF 

WEEK 

CALLS 

RECEIVED 

CALLS 

ANSWERED 

CALLS 

LOST 

PERCENT 

ANSWERED 

MONDAY 

7022 

5507 

1515 

78.4 

TUESDAY 

6910 

5715 

1195 

82.7 

WEDNESDAY 

6799 

5761 

1038 

84.7 

THURSDAY 

5801 

5183 

618 

89.3 

FRIDAY 

6051 

5581 

470 

92.2 

SATURDAY 

5006 

4299 

707 

85.9 

SUNDAY 

4304 

4105 

199 

95.4 

OVERALL 

AVERAGE 

5985 

5164 

821 

86.3 

OVERALL 

STANDARD 

DEVIATION 

957 

635 

420 

X 

Source:  WMATA  ACD  Readings 
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1150.  Calls  answered  daily  ranged  from  3526  to  7214,  with  an 
average  of  5164  and  a standard  deviation  of  801.  Thus,  both 
total  daily  customer  demand  and  total  daily  calls  answered  are 
approximately  1000  less,  on  average,  than  their  pre-implementa- 
tion levels.  Explanations  for  this  discrepancy  might  include 
seasonal  variations  in  demand,  varying  agent  work  forces,  and 
variations  in  average  agent  experience. 

From  Table  4-8,  it  can  be  seen  that  the  daily  post- 
implementation patterns  of  caller  demand  and  system  performance 
are,  with  several  minor  variations,  very  similar  to  the  pre- 
implementation patterns  described  above.  Figure  4-3  shows 
these  daily  patterns  expressed  as  percentages  of  total  average 
weekly  calls,  with  the  similar  percentages  from  Figure  4-1 
superimposed  for  comparison.  It  can  be  seen  from  this  Figure 
that  demand  again  peaks  on  Monday  at  nearly  17%  of  total  weekly 
demand  and  declines  throughout  the  week,  with  a slight  upward 
trend  on  Friday,  to  just  above  10%  on  Suriday.  From  Monday 
through  Friday,  calls  answered  range  between  13  and  14  per 
cent  of  total  weekly  demand,  while  on  weekends,  this  percentage 
is  reduced  to  between  9 and  10  per  cent. 

These  ACD  data  indicate  that  the  Information  Center  handled 
incoming  demand  best  on  Thursdays,  Fridays,  and  Sundays,  averaging 
just  over  90%  of  calls  on  these  days.  On  Wednesdays  and 
Saturdays,  this  efficiency  fell  to  around  85%,  while  the  peak 
demands  of  Mondays  and  Tuesdays  still  presented  a problem, 
with  approximately  80%  of  calls  being  answered  on  these  days. 

4.8  AIDS  DAILY  LOG  FILE  STATISTICS 

An  important  component  of  after-implementation  data  was 
the  daily  log  file  of  computer  transactions  automatically  kept 
by  the  AIDS.  These  data  present  an  accurate  and  comprehensive 
picture  of  agent  computer  usage.  Taken  in  the  context  of  the 
after-implementation  ACD  statistics,  they  provide  a measure  of 
the  extent  to  which  agents  are  selecting  computer  data  retrieval 
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PERCENT  OF  TOTAL  WEEKLY  DEMAND 


FIGURE  4-3 


DAILY  VARIATION  IN  CALLS  RECEIVED  AND  CALLS  ANSWERED 

AFTER  IMPLEMENTATION 

(BEFORE-IMPLEMENTATION  DATA  SHOWN  IN  DOTTED  LINES) 


\ 

\ 


DAY  OF  WEEK 

Source:  WMATA  ACD  Readings 


44 


over  manual  mode  methods.  Daily  computer  log  files  covering 
the  period  from  October  1 to  November  15,  1981  were  made 
available  to  Wilson  Hill  for  evaluation.  From  these  files,  a 
number  of  interesting  characteristics  of  AIDS  were  determined. 
These  usage  characteristics  are  described  below; 

• Agents  chose  to  use  computer  transactions  for  data 
retrieval  about  eight  to  nine  times  per  hour  logged-on, 

on  average,  during  the  period  for  which  data  was  ccllected. 
Average  computer  use  is  highest  during  the  Monday-Tuesday- 
VJednesday  demand  peaks,  and  declines  somewhat  through  the 
remainder  of  the  week.  Table  4-9  displays  average  total 
keyed  functions  per  hour  logged-on  by  day  of  the  week. 

• A fair  number  of  agent  data  inputs  result  in  error  messages 
or  prompts  from  the  AIDS  computer.  As  discussed  previously, 
these  "errors"  can  be  attributed  to  agent  misspellings  or 
input  format  errors,  or  to  caller  misinformation.  As  can 

be  seen  in  Table  4-9,  these  errors  occurred  at  the  rate 
of  approximately  four  per  hour  logged-on,  on  average. 

Thus,  valid  transactions  which  yielded  useful  data  to  the 
agent  (and  the  caller)  occurred  at  a rate  of  just  under 
five  per  hour,  on  average.  This  implies  that  43%  of  all 
data  entries,  or  one  keyed  transaction  out  of  every  2.31 
entries,  results  in  a system  error  response.  While  this 
percentage  may  appear  to  be  unacceptably  high,  it  should 
be  emphasized  that  callers  account  for  some  errors.  With 
increasing  agent  experience,  it  should  be  expected  that 
this  "error"  rate  will  decrease. 

• Table  4-10  displays  percentages,  averaged  by  week  of  the 
data  collection  period,  of  total  valid  AIDS  functions  for 
each  of  the  four  function  types  (itinerary,  schedule, 
route,  and  fare).  From  this  table,  it  is  evident  that 
the  itinerary  function  is  by  far  the  most  popular  with 
callers  and  agents,  accounting  for  almost  84%  of  all 
transactions.  The  schedule  function  is  the  next  most 
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TABLE  4-9 


CHARACTERISTICS  OF  AIDS  USAGE 
(AVERAGES  FOR  6-WEEK  PERIOD,  FIGURES  SHOWN 
PER  HOUR  LOGGED-ON) 


DAY 

OF 

WEEK 

TOTAL 

KEYED 

FUNCTIONS 

TOTAL 

"ERRORS" 

REPORTED 

TOTAL 

VALID 

TRANSACTIONS 

RATIO  OF 
VALID/ 
TOTAL 

MONDAY 

9.37 

3.97 

5.40 

57.6% 

TUESDAY 

9.89 

3.54 

6.35 

64.2% 

WEDNESDAY 

9.70 

3.31 

5.89 

60.7% 

THURSDAY 

7.82 

3.27 

4.55 

58.2% 

FRIDAY 

8.43 

3.40 

5.03 

59.7% 

SATURDAY 

8.67 

3.80 

4.87 

56.2% 

SUNDAY 

7.40 

3.64 

3.76 

50.8% 

OVERALL 

AVERAGE 

8.69 

3.72 

4.97 

56.7% 

OVERALL 

STANDARD 

DEVIATION 

1.89 

0.72 

0.63 

6.9% 

Source:  AIDS  Log  Files 
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TABLE  4-10 


PERCENTAGES  OF  AIDS  FUNCTIONS  PROCESSED  BY  TYPE 
(AVERAGES  FOR  EACH  WEEK  OF  DATA  COLLECTION) 


WEEK 

(1981) 

FUNCTION  TYPE 

ITINERARY 

SCHEDULE 

ROUTE 

FARE 

Oct.  5-11 

85 

11 

0 

4 

Oct.  12-18 

81 

14 

1 

4 

Oct.  19-25 

84 

11 

0 

5 

Oct.  2 5 -Nov.  1 

84 

10 

0 

6 

Nov.  2-8 

85 

10 

0 

5 

Nov.  9-15 

85 

9 

0 

6 

OVERALL 

AVERAGE 

83.61 

11.08 

0 .35 

5.15 

ON  AVERAGE, 

ONE  CALL  IN 
EVERY 

1.20 

9.00 

285.70 

19.41 

Source:  AIDS  Log  Files 


frequently  used,  comprising  another  11%  of  all  transactions. 
Fare  requests  account  for  an  additional  5%,  while  route 
description  transactions  are  almost  never  selected  by  the 
agents  (once  out  of  every  286  transactions,  on  average). 

This  breakdown  is  rather  to  be  expected  since  it  reflects 
the  relative  frequencies  of  inquiry  types  received  overall, 
and  since  itinerary  requests  require  the  most  manual 
research  on  the  part  of  the  agent,  while  manual  schedule 
references  are  a relatively  straightforward  and  speedy 
process.  Agents  who  have  undergone  the  full  WMATA  training 
program  (all  but  four  of  the  agents  during  the  data  collection 
period)  would  be  expected  to  know  most  route  description 
and  fare  data  from  memory. 

By  comparing  computer  records  to  ACD  readings,  it  is 
possible  to  make  some  qualified  statements  concerning  the 
percentage  of  total  calls  answered  for  which  AIDS  trans- 
actions are  used  for  data  retrieval.  The  problem  in 
reporting  this  percentage  is  that  the  computer  records 
data  only  by  transaction  and  not  by  call,  as  does  the  ACD 
system.  Although  the  great  majority  of  calls  involve 
only  one  AIDS  transaction,  it  is  quite  possible  that  a 
multiple-query  call  might  cause  the  agent  to  process  more 
than  one  valid  AIDS  transaction.  Making  the  imperfect 
assumption,  then,  that  one  valid  computer  transaction 
equals  one  call  processed,  an  upper  bound  can  be  determined 
for  the  percentage  of  calls  which  prompt  AIDS  usage. 

These  percentages  are  presented  in  Table  4-11.  From  this 
table  it  is  evident  that  AIDS  is  used  more  frequently 
during  the  peak  demand  days  of  the  week  (Monday,  Tuesday 
and  Wednesday) , during  which  a maximum  of  between  25  and 
30  per  cent  of  all  calls  prompt  computer  transactions. 

Computer  usage  tapers  off  through  the  rest  of  the  week  to 
a maximum  of  approximately  17%  of  all  calls  on  weekends. 
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TABLE  4-11 


RATIO  OF  VALID  AIDS  TRANSACTIONS  TO  TOTAL  CALLS  ANSWERED 

(PERCENT) 


WEEK 

(1981) 

DAY 

OF  WEEK 

MON 

TUE 

WED 

THU 

FRI 

SAT 

SUN 

Oct.  19-25 

32 

30 

28 

24 

22 

23 

20 

Oct.  26  -,  Nov.  1 

19 

30 

27 

17 

23 

15 

17 

Nov.  2-8 

30 

28 

23 

28 

14 

13 

17 

Nov.  9-15 

30 

19 

16 

25 

17 

17 

16 

AVERAGE 

28 

27 

24 

24 

19 

17 

18 

Source 


AIDS  Log  File 


4.9  AFTER-IMPLEMENTATION  CALL  MONITORING 

Included  in  after- implementation  data  collection  efforts 
was  an  intensive  three-week  session  of  call  monitoring.  This 
monitoring  was  accomplished  using  the  same  techniques  as  had 
been  used  previously  before  implementation.  Major  objectives 
of  this  monitoring  program  were  to: 

• verify  the  general  accuracy  of  computer  log  file  data; 

• supplement  computer  log  file  data  concerning  call  types 
received  and  agent  performance  in  answering  them; 

• provide  measurements  for  data  retrieval  and  total  call 
times,  items  which  were  explicitly  missing  from  both  ACD 
and  computer  log  file  data. 

Data  items  recorded  in  this  after  implementation  call 
monitoring  effort  included  the  following: 

• Day  and  date; 

• ID  number  of  operator  monitored,  and  whether  or  not  the 
operator  had  received  AIDS  training; 

• Call  type  (itinerary,  schedule,  route  or  fare); 

• Data  retrieval  time  (the  elapsed  time  between  the  end  of 

the  caller  query  and  the  beginning  of  the  agent  response) ; 

• Mode  of  data  retrieval  selected  by  the  operator  (manual 
only,  AIDS  only,  memory,  or  a mixture  of  the  above); 

• Total  call  time  (the  elapsed  time  from  the  operator's 
initial  greeting  to  final  caller  disconnect) ; 

• Qualitative  observations  regarding  any  perceived  call 

circumstances  that  might  have  affected  other  data  collected. 
Calls  were  monitored  from  November  23  to  December  11, 

1981.  During  this  time,  a total  of  1,341  calls  were  success- 
fully documented.  From  this  documentation,  frequencies  of 
important  variables  were  calculated  and  cross-tabulations  were 
produced  for  certain  data  items  of  interest.  These  results 
are  described  below: 

• Call  types  recorded  were  generally  consistent  with  pre-implem 
entation  monitoring  results.  Once  again  886,  or  66%  of 
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all  calls  monitored,  involved  itinerary  requests.  An 
additional  373,  or  27%  of  the  total,  involved  schedule 
information  queries,  while  fare  queries  were  mentioned  in 
81,  or  6%  of  all  calls.  Route  and  other  queries  were 
mentioned  only  in  one  instance  or  less  than  1%  of  the 
calls  monitored. 

Agents  used  manual  data  retrieval  exclusively  in  699,  or 
50.6%  of  all  calls  monitored.  AIDS  was  used  exclusively 
in  handling  an  additional  316  calls,  or  23.6%  of  all 
calls.  Agent  memory  (no  reference  to  either  AIDS  or 
manual  materials)  was  used  in  188,  or  14%  of  all  calls, 
while  agents  used  a mixture  of  the  retrieval  modes  in  the 
remaining  158  instances,  or  11.8%  of  the  total.  Since 
about  half  of  these  mixed-mode  retrievals  involved  AIDS, 
the  retrieval  breakdown  is  consistent  with  the  25-30% 

AIDS  use  breakdown  derived  from  comparing  computer  log 
and  ACD  data. 

Computer  retrieval  times  recorded  in  the  monitoring  process 
averaged  between  15  and  20  seconds,  although  occasionally 
longer  times  were  reported.  This  sample,  however,  was 
generated  during  a peak  period  of  caller  input  and  AIDS 
use  and  therefore  can  be  considered  an  "upper  limit"  on 
AIDS  processing  time. 

Total  call  times  observed  in  the  after  monitoring  effort 
are  relatively  similar  to  those  observed  pre- implementation 
A total  of  611  calls,  or  45.6%,  lasted  one  minute  or 
less.  An  additional  361,  or  26.9%,  had  durations  between 
one  and  two  minutes.  A total  of  194  calls,  or  14.5%, 
took  between  two  and  three  minutes  to  complete,  while  86, 
or  6.4%,  lasted  between  three  and  four  minutes.  Only  89, 
or  6.6%,  lasted  over  4 minutes.  This  frequency  distri- 
bution is  illustrated  in  Figure  4-4;  it  is  compared  in 
this  figure  with  the  pre- implementation  distribution 
shown  previously  in  Figure  4-2. 
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FIGURE  4-4 


FREQUENCY  DISTRIBUTION  OF  CALL  TIMES, 
BEFORE  (DOTTED  LINE) 

AND  AFTER  (SOLID  LINE)  IMPLEMENTATION 


Source:  WHA  Call  Monitoring 
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• A cross-tabulation  was  made  of  call  times  against  retrieval 
mode  used.  This  tabulation  is  shown  in  Table  4-12.  As 
can  be  seen  from  this  cross-tabulation,  manual  retrieval- 
only  calls  are  generally  of  short  durations;  85%  of  these 
calls  take  less  than  two  minutes  to  complete,  while  61% 
take  less  than  one  minute.  AIDS-only  and  mixed-mode 
calls  have  flatter  time  distributions,  and  average  around 
two  minutes  in  duration.  As  might  be  expected,  the  great 
majority  (77%)  of  memory-only  calls  take  less  than  one 
minute,  since  they  require  no  data  retrieval  time. 

• Table  4-13  depicts  a cross-tabulation  of  query  type  versus 
mode  of  data  retrieval  chosen.  As  can  be  seen  from  this 
table,  the  vast  majority  (85%)  of  schedule  queries  prompt 
agents  to  use  manual  retrieval.  On  the  other  hand,  84% 

of  all  AIDS-only  calls  are  itinerary  requests. 

• In  Table  4-14,  a cross-tabulation  of  call  time  vs.  query 
type  is  presented.  This  table  reinforces  the  finding 
that  schedule  requests  are  generally  handled  in  rapid 
fashion,  while  itinerary  and  fare  requests  usually  take 
longer . 

4.10  AFTER  IMPLEMENTATION  AGENT  QUESTIONNAIRE 

The  design  of  the  after-implementation  agent  question- 
naire, which  was  administered  during  November-December  1981, 
was  very  similar  to  that  of  the  before-implementation  question- 
naire. A total  of  12  questions  were  asked  of  agents  across  a 
broad  range  of  topics  regarding  the  impacts  of  AIDS  on  their 
work.  Of  these,  eleven  were  multiple-choice  and  three  provided 
blanks  for  open-ended  responses  on  the  part  of  the  agents.  A 
total  of  26  questionnaires  were  returned  for  analysis.  Fre- 
quencies for  each  response  as  well  as  several  cross-tabulations 
were  produced;  results  are  presented  below. 
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TABLE  4-12 


CROSS -TABULATION  OF  CALL  TIME  VS  DATA  RETRIEVAL  MODE 
AFTER  IMPLEMENTATION  CALL  MONITORING 


CALL 

(SECO 

m TMr* 

MODE  OF  DATA  RETRIEVAL 

TOTAL 

1 m£i 

NDS) 

MANUAL 

ONLY 

AIDS 

ONLY 

AGENT 

MEMORY 

MIXTURE 
OF  MODES 

0 - 

30 

180 

5 

82 

7 

274 

30  - 

60 

235 

21 

63 

18 

337 

60  - 

90 

115 

63 

20 

22 

220 

90  - 

120 

51 

56 

11 

23 

141 

120  - 

180 

66 

88 

8 

32 

194 

180  - 

240 

19 

37 

2 

28 

86 

240  - 

300 

8 

26 

2 

12 

48 

300  - 

360 

2 

14 

0 

7 

23 

Over 

360 

3 

6 

0 

9 

18 

TOTAL 

679 

316 

188 

158 

1341 

PERCENT 

OF 

TOTAL 

50.6 

23.6 

14.0 

11.8 
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TABLE  4-13 


CROSS -TABULATION  OF  QUERY  TYPE  VS  RETRIEVAL  MODE  CHOSEN 
AFTER  IMPLEMENTATION  CALL  MONITORING 


QUERY 

TYPE 

MODE  OF  DATA  RETRIEVAL 

TOTAL 

MANUAL 

ONLY 

AIDS 

ONLY 

AGENT 

MEMORY 

MIXTURE 
OF  MODES 

ITINERARY 

325 

264 

164 

133 

886 

SCHEDULE 

316 

31 

14 

12 

373 

ROUTE 

0 

1 

0 

0 

1 

FARE 

38 

20 

10 

13 

81 

TOTAL 

679 

316 

188 

158 

1341 
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TABLE  4-14 


CROSS-TABULATION  OF  CALL  TIME  VS  QUERY  TYPE 
AFTER  IMPLEMENTATION  CALL  MONITORING 


CALL 

TIME 

(SECONDS) 

QUERY  TYPE 

TOTAL 

ITINERARY 

SCHEDULE 

ROUTE 

FARE 

0 - 

30 

116 

140 

0 

17 

273 

30  - 

60 

195 

144 

0 

25 

364 

60  - 

90 

140 

43 

0 

13 

196 

90  - 

120 

115 

25 

0 

7 

147 

120  - 

180 

156 

12 

0 

13 

181 

180  - 

240 

77 

6 

0 

5 

88 

240  - 

360 

68 

2 

1 

1 

72 

Over 

360 

19 

1 

0 

0 

20  . 

TOTAL 

886 

373 

1 

81 

1341 

PERCENT 

OF 

TOTAL 

66.07 

27.81 

.08 

6.04 
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To  the  question,  "How  long  have  you  worked  as  an  infor- 
mation agent?",  one  agent  replied  "7-12  months",  7 replied 
"one  to  two  years",  4 replied  "two  to  three  years",  and 
14  replied  "over  three  years".  Thus  the  respondents 
averaged  at  least  2.2  years  on  the  job.  These  experience 
responses  are  compared  with  before-implementation  survey 
responses  to  the  same  question  in  Table  4-15.  From  this 
table,  it  can  be  seen  that  the  respondents  to  the  after- 
implementation survey  were  generally  more  experienced 
than  the  before-implementation  group. 

In  response  to  the  question  "How  long  have  you  been  using 
AIDS?"  14  replied  "6  or  more  months",  9 agents  replied  "3 
to  6 months",  and  3 responded  "less  than  3 months".  Thus 
the  sample  was  also  well  experienced,  for  the  most  part, 
in  AIDS  use. 

To  the  question  "What  is  your  hourly  call  count  under 
busy  conditions?"  Four  agents  chose  the  response  "16-20", 
nine  chose  "21-25",  10  chose  "26-30",  and  three  chose 
"over  30".  This  distribution  works  out  to  an  average  of 
at  least  23  calls  per  hour.  This  distribution  of  responses 
is  quite  similar  to  that  of  the  before-implementation 
survey,  as  shown  in  Table  4-16. 

Agent  perceptions  as  to  how  AIDS  use  affected  their  call 
counts  were  of  particular  interest.  In  the  questionnaire, 
a total  of  12  agents  responded  that  AIDS  use  reduced 
their  call  counts;  8 agents  responded  that  AIDS  increased 
theirs,  and  6 more  agents  responded  that  their  call  counts 
remained  the  same.  This  rather  negative  result  came  as 
something  of  a surprise  to  WMATA  supervisory  and  manage- 
ment personnel,  who  had  well-documented  evidence  through 
routine  call  monitoring  and  spot  performance  checks  on 
agents  that  call  counts  did  initially  drop  following  AIDS 
training,  but  soon  regained  normal  or  higher  levels  once 
computer  proficiency,  including  typing  skills  and  mani- 
pulation of  keyboard  functions,  was  achieved.  Analyzing 
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TABLE  4-15 


COMPARISONS  OF  AGENT  RESPONSES  TO  SURVEY  QUESTIONS 
REGARDING  JOB  EXPERIENCE 

BEFORE  AND  AFTER  AIDS  IMPLEMENTATION 


EXPERIENCE 
AS  AN  AGENT 

TOTAL  RESPONSES 

% OF  TOTAL 

BEFORE 

AFTER 

BEFORE 

AFTER 

0 - 6 MO. 

3 

0 

7 

0 

7 - 12  MO. 

8 

1 

19 

4 

1-3  YRS. 

13 

7 

31 

27 

3-5  YRS. 

9 

4 

21 

15 

OVER  5 YRS. 

9 

14 

21 

54 
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TABLE  4-16 


COMPARISONS  OF  AGENT  RESPONSES  TO  SURVEY  QUESTIONS 
REGARDING  HOURLY  CALL  COUNTS 

BEFORE  AND  AFTER  IMPLEMENTATION 


CALL  COUNT 
PER  HOUR 
AS  REPORTED 
BY  AGENT 

TOTAL  RESPONSES 

% OF  TOTAL 

BEFORE 

AFTER 

BEFORE 

AFTER 

0-15 

2 

0 

5 

0 

16  - 20 

5 

4 

12 

15 

21  - 25 

15 

9 

36 

35 

26  - 30 

12 

10 

29 

38 

OVER  30 

8 

3 

19 

12 
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the  12  negative  responses  further,  it  can  be  seen  that 
half  of  these  agents  had  been  using  AIDS  for  less  than  6 
months.  Additionally  half  (although  not  the  same  six) 
responded  that  they  used  the  system  only  "a  few  times  a 
day".  Thus,  the  agents  responding  to  the  after-implemen- 
tation survey  may  not  have  been  entirely  representative 
of  the  total  agent  work  force  as  regards  the  influence  of 
AIDS  on  productivity. 

To  the  question  "How  often  do  you  use  AIDS?",  8 agents 
responded  "few  times  a day",  15  responded  "over  1/2  my 
calls",  and  2 responded  "almost  every  call". 

Nine  agents  chose  the  response  "excellent"  to  rate  their 
ability  using  AIDS.  An  additional  15  rated  their  ability 
as  "good",  while  2 chose  the  "fair"  response.  None  of 
the  agents  rated  themselves  "poor"  in  this  category. 

Agents  had  a fairly  good  opinion  of  AIDS  training.  A 
total  of  22  agents  rated  their  training  as  "just  right", 
while  only  three  claimed  that  there  was  "much  more  training 
than  was  necessary",  and  only  one  selected  the  response 
"training  was  not  enough". 

In  rating  their  overall  job  satisfaction,  six  agents 
chose  "excellent"  as  a response,  while  16  chose  "good" 
and  four  chose  "fair".  None  of  the  respondents  chose  the 
"poor"  response.  These  responses  are  very  similar  to 
those  of  the  before- implementation  survey,  as  shown  in 
Table  4-17. 

To  the  question  "In  what  way  has  learning  to  use  AIDS 
influenced  your  job  satisfaction?"  agents  gave  the  system 
an  overwhelming  vote  of  confidence.  A total  of  19  agents 
chose  "in  a positive  way",  six  more  chose  "no  influence", 
and  none  chose  "in  a negative  way".  These  responses  are 
compared  with  agent  perceptions  of  AIDS  influence  before 
implementation  in  Table  4-18.  From  this  table,  it  can  be 
seen  that  "negative"  feelings  towards  AIDS  have  been 
reduced  following  implementation. 
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TABLE  4-17 


COMPARISONS  OF  AGENT  RESPONSES  TO  SURVEY  QUESTIONS 
REGARDING  JOB  SATISFACTION 

BEFORE  AND  AFTER  IMPLEMENTATION 


AGENT  SELF- 
RATING  OF 
SATISFACTION 

TOTAL  RESPONSES 

% OF 

TOTAL 

BEFORE 

AFTER 

BEFORE 

AFTER 

EXCELLENT 

7 

6 

17 

23 

GOOD 

24 

16 

57 

62 

FAIR 

10 

4 

24 

15 

POOR 

1 

0 

2 

0 
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TABLE  4-18 


COMPARISONS  OF  AGENT  RESPONSES  TO  QUESTIONS 
REGARDING  AIDS  INFLUENCE  ON  THEIR  JOBS 

BEFORE  AND  AFTER  IMPLEMENTATION 


IMPACT 
CATEGORY 
CHOSEN 
BY  AGENT 

TOTAL  RESPONSES 

% OF  TOTAL 

BEFORE 

AFTER 

BEFORE 

AFTER 

POSITIVE 

34 

19 

83 

76 

NO  CHANGE 

2 

6 

5 

24 

NEGATIVE 

5 

0 

12 

0 
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To  the  question  "What  aspect  of  AIDS  causes  you  the 
greatest  problem?",  agents  were  provided  four  choices 
plus  a blank  labeled  "other"  for  agents  to  supply  their 
own  comments.  Two  agents  chose  the  response  "choosing 
which  function  to  use",  three  agents  chose  "typing  in 
caller  information",  and  12  chose  "completeness/accuracy 
of  responses".  A total  of  7 agents  provided  comments 
under  "other".  Of  these,  four  mentioned  problems  with 
"sectors"*  and  three  criticized  the  slowness  of  system 
responses. 

Agents  were  asked  to  compare  AIDS  with  manual  information 
processing  across  a broad  range  of  categories.  Agent 
responses  are  summarized  in  Table  4-19.  From  this  table, 
it  can  be  seen  that  agents  are,  on  the  whole,  quite  favor- 
ably disposed  to  the  system.  The  AIDS  system  was  rated 
"better"  in  97  agent  responses,  or  53.3%  of  the  total, 
and  "about  the  same"  in  an  additional  67  responses,  or 
36.8%.  Only  13  responses  characterized  AIDS  as  "worse" 
than  manual  (7.1%  of  the  total).  Particularly  strong 
positive  showings  for  AIDS  came  in  the  categories  of 
"detail  of  information"  and  "making  job  easier".  The 
strongest  negative  comparison  came  under  the  category 
"speed  of  retrieving  information",  in  which  16  agents 
felt  manual  was  about  the  same  or  better  than  AIDS. 

For  the  question  "How  can  AIDS  be  improved  to  help  you  in 
your  work?"  only  a blank  was  provided  for  open-ended 
agent  responses.  A number  of  interesting  suggestions 
were  made  by  agents,  highlighting  some  of  the  weaknesses 
of  the  system  in  its  present  form. 


*The  AIDS  geographic  data  base  is  divided  into  sectors  based 
on  political  boundaries  in  the  D.C.  metropolitan  area.  The 
"Sector"  input  must  be  correct  before  the  computer  is  able  to 
locate  address  inputs  This  "Sector"  input  is  a leading  cause 
of  agent  "errors." 
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TABLE  4-19 


AGENT  COMPARISONS  OF  AIDS  VS  MANUAL  DATA  RETRIEVAL  METHODS 
AFTER  IMPLEMENTATION  AGENT  SURVEY 


COMPARISON 

CATEGORY 

AGENT  RESPONSES 

AIDS  BETTER 
THAN  MANUAL 

AIDS  WORSE 
THAN  MANUAL 

AIDS  ABOUT  THE 
SAME  AS  MANUAL 

SPEED 

9 

6 

10 

ACCURACY 

11 

2 

11 

DETAIL 

20 

1 

5 

JOB  FATIGUE 

9 

1 

15 

MULTIPLE  QUERY 

14 

2 

9 

MAKE  JOB  EASIER 

20 

0 

6 

CALLER  SATISFACTION 

14 

1 

11 

TOTAL 

97 

13 

67 

% OF  ALL 

RESPONSES 

53.3 

7.1 

36.8 
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The  most  frequent  agent  suggestion  was  that  the 
speed  of  the  system  be  improved,  especially  for  schedule 
requests.  From  the  call  monitoring  data,  it  would  seem 
as  though  agents  presently  handle  most  schedule  requests 
manually,  because  manual  retrieval  is  in  most  instances 
faster . 

Other  agents  suggested  expanding  the  "time  window" 
of  schedule  function  responses.  At  present,  all  scheduled 
runs  for  a given  route  are  displayed  on  the  response 
screen  for  the  two-hour  period  beginning  one  hour  before 
and  ending  one  hour  after  the  reference  time  entered  by 
an  agent.  Agents  would  like  to  see  this  "window"  increased 
to  4 or  more  hours  if  possible,  or  for  the  function  to 
include  a "scrolling"  capacity  so  that  the  time  window 
could  be  manipulated  by  the  agent.  A related  suggestion 
was  for  the  development  of  a "last  bus"  function.  To 
determine  what  time  the  last  scheduled  bus  of  the  day 
passes  a given  location,  the  agent  must  guess  a time 
reference  point  which  is  likely  to  include  the  last  bus 
within  its  two-hour  "window".  For  example,  suppose  a 
given  bus  route  passes  a certain  scheduled  stop  at  35 
minutes  past  the  hour  every  hour  until  12:35  a.m.  If  an 
agent  entered  11:30  in  the  schedule  function  for  that 
particular  stop,  the  last  bus  at  12:35  would  not  appear 
on  the  response  screen.  With  the  long  headways  for  most 
runs  late  at  night,  a two-hour  "window"  is  considered  by 
most  agents  to  be  insufficient. 

Another  frequent  suggestion  was  for  development  of  a 
"nearest  stop"  function.  The  system  is  programmed  to 
search  for  stops  within  a 1/2  mile  of  a given  input  address. 
If  no  stops  are  found,  the  computer  produces  an  "error" 
message  and  the  agent  is  forced  to  remanipulate  the  address 
input.  Agents  feel  that  the  capacity  to  manipulate  the 
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system's  "searching  radius"  would  be  more  effective, 
especially  for  remote  suburban  addresses  where  stops  are 
likely  to  be  widely  spaced. 

The  present  system  of  landmarks  in  the  geographic 
data  base  was  the  subject  of  criticism  by  several  agents. 
Since  these  landmarks  were  added  incrementally  over  the 
course  of  system  implementation,  they  are  not  standardized, 
and  agents  must  therefore  memorize  their  correct  spelling 
(or  abbreviation)  to  ensure  recognition  and  verification 
by  the  system. 

A final  suggestion  was  made  concerning  development 
of  a "forcing"  function.  Using  such  a function,  an  agent 
could  "force"  the  computer  to  search  out  itineraries 
using  a stated  bus  line,  even  if  that  line  did  not  make 
the  minimum-time  itinerary.  Agents  occasionally  encounter 
customers  who  are  familiar  with  a certain  bus  line  or 
route  and  who  do  not  want  to  change  their  transit  habits 
even  if  such  a change  would  make  their  trip  shorter  or 
more  economical. 

Tables  4-20,  4-21  and  4-22  are  all  cross-tabulations 
which  were  produced  from  the  agent  questionnaire  results. 

Of  particular  interest  in  these  cross-tabulations  were 
the  rather  negative  responses  to  the  "AIDS  influence  on 
hourly  call  count"  question;  these  form  one  of  the 
variables  in  each  cross-tabulation. 

Table  4-20  presents  AIDS  influence  on  call  count 
versus  the  frequency  of  AIDS  use.  This  table  supports 
the  intuitive  notion  that  agents  who  felt  that  AIDS 
increased  their  call  count  tend  to  use  it  more  frequently 
than  those  who  did  not. 

Table  4-21  presents  overall  agent  experience  versus 
AIDS  influence  on  call  count.  From  this  table,  it  can 
be  seen  that  agents  with  less  experience  tended  to  feel 
that  AIDS  increased  their  call  counts,  while  more  experienced 
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TABLE  4-20 


CROSS -TABULATION  OF  AIDS  INFLUENCE  ON  CALL  COUNT 
VS  FREQUENCY  OF  AIDS  USE 
AFTER  IMPLEMENTATION  AGENT  SURVEY 


AIDS  INFLUENCE 
ON  CALL  COUNT 

FREQUENCY  OF  AIDS  USE 

TOTAL 

FEW  TIMES 
A DAY 

OVER  HALF 
OF  CALLS 

ALMOST 
EVERY  CALL 

REDUCTION 

6 

5 

1 

12 

ABOUT  THE  SAME 

1 

4 

1 

6 

INCREASE 

1 

6 

0 

7 

TOTAL 

8 

15 

2 

25 
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TABLE  4-21 


CROSS -TABULATION  OF  AGENT  JOB  EXPERIENCE 
VS  PERCEPTION  OF  AIDS  INFLUENCE  ON  CALL  COUNT 
AFTER  IMPLEMENTATION  AGENT  SURVEY 


AGENT  JOB 
EXPERIENCE 

PERCEPTION  OF  AIDS  INFLUENCE 
ON  CALL  COUNT 

REDUCTION 

SAME 

INCREASE 

7 - 12  MO. 

0 

0 

1 

1-2  YRS. 

1 

1 

5 

2-3  YRS. 

2 

2 

0 

OVER  3 YRS. 

9 

3 

2 

TOTAL 

12 

6 

8 

68 


TABLE  4-22 


CROSS -TABULATION  OF  AIDS  EXPERIENCE  VS  PERCEPTION  OF  AIDS 

INFLUENCE  ON  CALL  COUNT 

AFTER  IMPLEMENTATION  AGENT  SURVEY 


AIDS 

EXPERIENCE 

AIDS 

INFLUENCE 

ON  CALL  COUNT 

TOTAL 

REDUCTION 

SAME 

INCREASE 

0-3  MOS. 

1 

1 

1 

3 

3-6  MOS. 

5 

2 

2 

9 

OVER  6 MOS. 

6 

3 

5 

14 

TOTAL 

12 

6 

8 

26 
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agents  (with  3 or  more  years  on  the  job)  tended  towards 
the  opposite  opinion.  This  result  may  be  viewed  as 
rather  discouraging,  but  as  discussed  above,  survey 
respondents  might  not  be  wholly  representative  of  the 
agent  work  force. 

Table  4-22  plots  AIDS  experience  versus  AIDS  influence 
on  call  count.  From  this  table,  it  is  evident  that 
AIDS  experience  has  little  influence  on  perceptions  of 
call  count  influence. 

5.0  CONCLUSIONS 

5.1  BEFORE -AFTER  DATA  COMPARISONS 

The  various  quantitative  and  qualitative  data  collected 
at  WMATA  Telephone  Information  Section  and  detailed  in  the 
previous  chapter  permit  a number  of  "before-after”  comparisons 
to  be  made  concerning  caller  demands  and  agent  and  system 
performance  in  meeting  these  demands.  It  should  be  remembered, 
however,  that  AIDS  implementation  is  not  the  single  contributing 
factor  to  any  changes  that  might  be  observed  in  these  data.  The 
Telephone  Information  Section  is  not  a laboratory  in  which 
all  external  forces  can  be  controlled.  Three  other  major 
changes  have  also  had  an  influence  on  Section  operations; 

• The  Section  has  experienced  a relatively  high  turnover 
rate.  The  position  of  information  operator  is  considered 
entry-level  on  the  WMATA  pay  scale,  and  many  operators 
use  it  as  a stepping  stone  to  higher  paying  positions 
elsewhere  in  the  Authority.  This  turnover  has  had  the 
effect  of  lowering  the  average  job  experience  for  the 
Information  Section,  a factor  which  may  be  detrimental 

to  its  overall  performance  since  agents  and  management 
agree  that  productivity  increases  with  job  experience. 

• Customer  demand  is  inherently  variable,  and  exhibits 
seasonal  changes  over  the  course  of  the  year  in  much 
the  same  way  it  varies  frcm  day  to  day  over  the  course 
of  the  week. 
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• Implementation  of  AIDS  included  the  construction  of  an 
entirely  new  workspace  for  the  information  operators. 

WMATA  management  personnel  indicated  in  interviews  that 
system  performance  and  agent  job  satisfaction  were  en- 
hanced to  a considerable  degree  by  these  new  surroundings. 

Several  "before-after"  comparisons  have  already  been 
presented  in  Figures  4-3  and  4-4  and  Tables  4-15  through 
4-18.  The  discussion  below  presents  several  further  comparisons 
based  on  the  data  collected  before  and  after  implementation. 

• ACD  data;  In  the  before- implementation  period,  ACD 
readings  show  that  the  Information  Section  received  an 
average  of  6813  calls.  Of  these,  6162,  or  90.4%,  were 
answered  on  average.  In  the  after- implementation  period 
the  Section  received  an  average  of  5966  calls  and  answered 
an  average  of  5126,  or  85.9%  of  the  daily  total.  Thus 
overall  caller  demand  declined  12.4%  between  the  two 
periods,  while  calls  answered  declined  an  even  larger 
16.8%.  Overall  system  productivity  thus  declined  4.4%. 

This  reduction  might  be  explained  in  part  by  agent 
turnover  and  by  the  demand  variation  factors  described 
above.  It  might  also  be  explained  by  the  relative 
newness  of  the  system  to  agents  while  after- implementation 
performance  data  were  being  recorded.  It  is  probable 
that  overall  productivity  would  improve  with  increasing 
agent  familiarity  with  the  system.  Nevertheless,  demand 
patterns  over  the  course  of  the  week  remained  very 
stable  over  the  course  of  system  implementation,  as 
shown  previously  in  Figure  4-3.  Based  on  these  data,  the 
implication  is  that  little  has  been  accomplished  by 
system  implementation  towards  the  reduction  of  the 

"lost  call"  rate  of  10  - 15  percent  of  all  incoming 
calls. 

• The  first  and  second  columns  of  Table  5-1  present  data 
on  call  content  as  determined  by  call  monitoring  before 
and  after  implementation.  As  can  be  seen  from  this 

data,  percentages  of  various  call  types  remained  remarkably 
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TABLE  5-1 

CALL  TYPE  FREQUENCIES  BEFORE  AND  AFTER  AIDS  IMPLEMENTATION 


CALL 

TYPE 

FREQUENCY  OF  OCCURRENCE, 

% 

BEFORE 

IMPLEMENTATION 

AFTER 

IMPLEMENTATION 

COMPUTER 
LOG  FILE 

ITINERARY 

66 

66 

84 

SCHEDULE 

24 

27 

11 

ROUTE 

7 

0 

0 

FARE 

3 

7 

5 

Source : 


WHA  Call  Monitoring, 
AIDS  Log  File 
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constant  over  the  implementation  period.  It  is  interesting 
to  compare  these  data  with  call  type  percentages  handled 
by  the  computer,  as  shown  in  the  third  column  of  Table 
5-1.  These  computer  data  show  that  agents  use  the 
computer  for  a disproportionately  high  number  of  itinerary- type 
calls,  and  for  a correspondingly  lower  number  of  schedule- type 
call.  Call  monitoring  results  showed  that  schedule- type 
calls  were  generally  handled  fastest  manually  even 
after  implementation.  Hence,  these  data  are  consistent 
with  this  observation.  (See  Table  4-13.) 

Data  on  call  times  collected  before  and  after  imple- 
mentation are  presented  in  Table  5-2.  This  table  shows 
an  increase  of  3.5%  in  calls  answered  in  one  minute  or 
less,  although  the  percentage  of  calls  handled  in  three 
minutes  or  less  is  unchanged.  Considering  the  stated 
objective  of  system  implementation  to  increase  agent 
productivity,  this  result  is  rather  discouraging. 

Data  sources  consulted  as  reference  materials  were 
substantially  changed  by  AIDS  implementation.  Before 
and  after  implementation  data  concerning  these  sources 
are  presented  in  Table  5-3.  This  table  indicates  that 
the  frequency  of  "top  of  head"  responses  (no  reference 
materials  consulted)  remained  fairly  constant  at  approxi- 
mately 15%  of  total  calls  throughout  the  implementation 
period.  The  computer,  however,  has  replaced  manual 
data  for  between  25  and  30  percent  of  all  calls  referenced. 
Agent  questionnaire  results  before  and  after  implementation 
generally  indicate  a high  degree  of  agent  support  for  AIDS, 
particularly  in  the  area  of  "making  job  easier".  This 
support  is  tempered  somewhat  by  frustration  regarding 
response  speed,  especially  for  schedule- type  questions. 

Few  agents  reported  experiencing  problems  anticipated 
in  the  before  survey  in  adjusting  to  AIDS,  including 
typing  and  scanning  the  CRT  screens. 


73 


TABLE  5-2 


CALL  TIME  FREQUENCIES  BEFORE  AND  AFTER  AIDS  IMPLEMENTATION 
(PERCENT  OF  ALL  CALLS  ANSWERED) 


CALL  TIME 
(SECONDS) 

BEFORE 

IMPLEMENTATION 

AFTER 

IMPLEMENTATION 

DIFFERENCE 

0-60 

42.1 

45.6 

+ 3.5 

60  - 120 

31.7 

26.9 

-4.5 

120  - 180 

13.2 

14.5 

+1.3 

180  - 240 

5.4 

6.4 

+1.0 

OVER  240 

7.6 

6.6 

-1.0 

Source:  WHA  Call  Monitoring 
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TABLE  5-3 


DATA  SOURCES  CONSULTED  BY  AGENTS 
BEFORE  AND  AFTER  AIDS  IMPLEMENTATION 

(FREQUENCY  OF  USE,  PERCENT) 


DATA 

SOURCE 

BEFORE 

IMPLEMENTATION 

AFTER 

IMPLEMENTATION 

DIFFERENCE 

MANUAL 

ONLY 

85.4 

50.6 

-34 . 8 

COMPUTER 

ONLY 

★ 

23.6 

+ 23.6 

MIXTURE 
OF  SOURCES 

* 

11.8 

+ 11 . 8 

AGENT 

MEMORY 

14.6 

14.0 

-.6 

* Did  not  exist  before  implementation 


Source:  WHA  Call  Monitoring 
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5.2  SUCCESS  OF  AIDS  IMPLEMENTATION 

A review  of  the  above  comparisons,  supplemented  by 
other  performance  data  reported  in  the  previous  chapter, 
leads  to  generally  mixed  findings  concerning  the  success  of 
AIDS  implementation.  While  system  development  and  debugging 
took  longer  than  anticipated,  the  spirit  of  cooperation 
between  system  designers  and  system  users  throughout  this 
process  has  ensured  its  eventual  usefulness.  Without  this 
cooperation,  it  is  highly  unlikely  that  AIDS  would  be  used 
as  a data  source  with  nearly  the  frequency  that  it  is  today. 
Although  there  are  still  a few  minor  unresolved  problems,  it 
can  be  stated  that  AIDS  is  currently  being  used  as  system 
designers  and  WMATA  management  intended.  The  anticipated 
benefits  of  AIDS  implementation,  as  stated  earlier,  were: 

• improved  productivity  in  terms  of  agent  calls  per  hour 
answered  due  to  faster  data  retrieval; 

• increased  reliability  and  consistency  of  agent  responses; 

• a reduction  in  training  time  for  new  agents; 

• improved  capability  to  rapidly  incorporate  changes  in 
transit  agent  reference  data; 

• enhancement  of  the  agent's  overall  job  satisfaction; 

• expanded  capability  to  gather  statistics  concerning 
individual  agent  and  Information  Section  performance; 

• use  of  the  transit  system  data  base  by  other  WMATA 
departments  as  a planning  source. 

The  remainder  of  this  Section  discusses  the  achievement 
to  date  of  AIDS  deployment  in  each  of  these  anticipated 
benefit  areas.  As  might  be  expected,  data  collected  regarding 
each  area  has  occasionally  produced  ambiguous  results.  In 
the  following  paragraphs,  a number  of  conclusions  are  drawn 
concerning  anticipated  AIDS  benefits. 

• Agent  Productivity:  Data  collected  during  the  course 

of  the  evaluation  lead  to  conflicting  conclusions 
regarding  system  influences  on  agent  productivity. 
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Agents  were  evenly  divided  on  the  after-implementation 
questionnaire  regarding  AIDS'  influence  on  call  counts, 
and  a number  of  them  complained  about  system  response 
speed.  Furthermore,  ACD  data  revealed  an  overall  decrease 
in  Information  Section  productivity  (in  terms  of  calls 
lost)  post- implementation.  In  contrast,  agents  were 
almost  unanimous  on  the  after-implementation  survey  in 
the  opinion  that  AIDS  made  their  jobs  easier,  and  routine 
performance  tests  conducted  by  WMATA  management  reveal 
that  operators  using  AIDS  are  capable  of  answering 
calls  at  the  rate  of  30  to  40  per  hour,  a 30%  increase 
over  their  usual  manual  speed. 

WMATA  supervisory  and  management  personnel  were 
able  to  shed  some  light  on  the  causes  underlying  these 
conflicting  results.  It  appears  that  given  current 
system  response  speeds,  customer  schedule  questions  are 
still  handled  fastest  manually;  this  fact  is  supported 
by  the  after-implementation  call  monitoring  data  (see 
Table  4-13).  Thus,  agents  who  use  the  computer  for 
nearly  every  call  are  actually  less  productive  overall 
than  those  who  use  it  more  selectively  (i.e.,  who  handle 
schedule  requests  manually) . Varying  agent  experience 
levels  are  also  a contributing  factor  to  lower  post- 
implementation productivity,  since  newer  agents  tend  to 
rely  on  computer  or  manual  references  more  than  their 
more  experienced  counterparts,  who  rely  more  on  memory. 

The  Information  Section  experienced  considerable  turnover 
in  personnel  during  system  implementation.  Supervisory 
personnel  indicated  that  very  experienced  operators 
(having  10  or  more  years  on  the  job)  are  able  to  respond 
to  75%  of  their  calls  from  memory.  These  operators 
tend  to  use  AIDS  only  for  calls  requiring  construction 
of  long  or  complex  itineraries.  It  is  not  surprising, 
then,  that  the  more  experienced  operators  would  feel 
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that  using  the  computer  would  slow  them  down  in  most 
instances,  since  memory  responses  are  shorter  than  AIDS 
consultation,  but  would  still  be  of  the  opinion  that  it 
made  their  jobs  easier  (for  the  more  complex  questions). 

Supervisory  personnel  indicated  that  they  were 
initially  discouraged  by  agent  call  counts  following 
AIDS  training.  When  it  was  made  clear  to  the  agents 
that  using  AIDS  was  not  necessarily  the  most  useful  or 
efficient  way  to  respond  to  every  call,  productivity 
increased  considerably,  usually  to  levels  above  previous 
averages  for  manual  data  retrieval.  Management  and 
supervisors  both  felt  that  a 50-50  balance  between 
computer  and  manual  data  retrieval  produced  the  highest 
overall  call  count  for  the  average  agent.  Monitoring 
data  show  that  this  balance  has  not  yet  been  achieved; 
the  computer  is  currently  used  for  only  25-30%  of  all 
calls. 

Certain  system  improvements  are  currently  being 
contemplated  that  may  have  a strong  positive  impact  on 
the  ambiguous  productivity  results  discussed  above. 

They  are: 

Hardware  improvements  to  increase  system  capacity 
and  decrease  system  response  time. 

Improvements  to  the  "schedule"  function  which 
would  make  the  information  displayed  for  a typical 
response  much  more  comprehensive  than  the  infor- 
mation now  produced  and  comparable  to  that  currently 
available  from  a manual  reference. 

Response  Accuracy  and  Consistency:  The  agents  felt 

that  AIDS  responses  were  more  accurate  and  had  a greater 
level  of  detail  than  manual  responses  (see  Table  4-15). 
Supervisory  personnel  stated  that  the  problem  of  misinfor- 
mation, while  affecting  a relatively  small  percentage 
(1-2%)  of  all  calls  answered,  was  greatest  with  inex- 
perienced operators  using  manual  materials  only. 
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Supervisors  indicated  that  they  felt  much  more  com- 
fortable when  these  operators  consulted  AIDS. 

A feature  of  AIDS  that  helps  ensure  the  nearly  100% 
accuracy  of  its  data  base  is  the  built-in  error-reporting 
function.  If  an  operator  receives  a response  that  he/she 
deems  questionable,  activating  the  reporting  function 
will  cause  the  response  to  be  listed,  along  with  the 
agent's  ID  number,  on  a daily  "trouble  report".  A review 
of  this  report  by  agents  and  management  helps  to  locate 
inconsistencies  or  errors  in  the  data  base,  as  well  as 
to  identify  areas  in  which  an  agent's  training  might  be 
deficient.  It  cannot  be  overemphasized  that  this  enhance- 
ment process  is  an  interactive  one  in  which  agents  and 
Information  Section  management  are  active  participants. 
This  interaction  not  only  results  in  greater  accuracy  of 
the  data  base  and  a more  workable  system,  but  also  leads 
to  increased  confidence  on  the  part  of  the  agents  that 
they  are  personally  responsible  for  system  accuracy. 

Agent  Training:  Interviews  with  Information  Section 

management  indicated  that  the  the  anticipated  benefit  of 
reduced  training  time  for  operators  does  not  appear  to 
be  achievable,  based  on  results  with  four  new  agents. 

These  agents  received  a two-week  course  which  included 
computer  manipulation  practice,  communications  skills 
development,  and  some  rudimentary  background  in  use  of 
manual  references  (they  already  possessed  typing  skills) . 
They  were  not  given  the  intensive  geographic  and  transit 
system  practice  which  forms  a major  part  of  the  tradi- 
tional 6-week  manual  training  period.  After  two  weeks, 
these  agents  were  producing  "acceptable"  cal]  counts  of 
80-90  per  day,  and  it  seemed  to  management  as  though  the 
training  reduction  benefit  had  been  achieved. 
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Reports  from  supervisory  personnel,  however,  soon 
produced  strong  evidence  to  the  contrary.  Without  the 
geographic  and  transit  system  practice  of  the  standard 
training  course,  these  new  agents  were  unable  to  provide 
details  to  callers  concerning  landmarks  to  use  as  reference 
points  on  their  journeys.  Even  worse,  they  were  unable 
to  make  interpretive  judgements  as  to  which  of  the  four 
feasible  alternative  responses  routinely  produced  by 
AIDS  best  suited  the  caller's  request.  In  the  opinion 
of  the  supervisors,  they  became  too  dependent  on  the 
computer  for  "simple"  questions,  such  as  fare  and  schedule 
requests,  which  other  agents  would  have  handled  manually 
or  from  memory.  To  remedy  these  observed  deficiencies 
in  their  performance,  these  agents  were  later  given 
additional  training  emphasizing  map  work  and  manual 
schedule  practice.  With  this  additional  training,  these 
agents  were  better  able  to  picture  routes,  intersections, 
and  landmarks,  and  achieved  full  proficiency  (150-170 
calls  per  day) . 

The  net  result  of  this  experience  was  to  point  out  ■ 
that  agents  with  AIDS  training,  but  lacking  a good  foundation 
in  manual  practice  and  geographic  data  manipulation,  are 
of  marginal  value.  While  these  operators  could  be  used 
as  a stopgap  measure  during  periods  of  critical  manpower 
shortage,  their  long-term  usefulness  is  limited. 

Updating  of  Data  Base:  It  is  important  to  recognize 

that  the  geographic  and  transit  data  bases,  once  established, 
are  not  static.  Rather,  they  must  be  constantly  changed 
to  reflect  changes  in  local  land  use  and  in  transit 
service.  Making  these  changes  requires  the  full-time 
effort  of  one  WMATA  employee.  They  are  made  so  frequently 
that  the  AIDS  geographic  data  base  is  becoming  recognized 
as  the  most  accurate  address  directory  available  for  the 
Washington,  D.C.  metropolitan  area. 
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While  theoretically  the  transit  data  base  updates 
could  be  made  in  the  same  centralized  fashion  as  the 
geographic  updates  and  copies  printed  by  the  AIDS  computer 
for  manual  use,  this  does  not  happen  in  practice.  Instead, 
word  processing  equipment  in  the  Scheduling  Department 
of  WMATA  (installed  during  AIDS  implementation)  is  used 
to  produce  the  hard  copy  manual  data  base  materials,  and 
several  clerks  work  full-time  to  keep  these  manual  references 
updated.  So  long  as  implementation  of  AIDS  is  mixed 
(that  is,  manual  as  well  as  computer  data  referencing  is 
available)  there  will  be  little  cost  savings  from  centralized 
data  base  updating.  No  shift  away  from  the  mixed  operation 
is  contemplated  in  the  near  future.  Thus  actual  cost 
reductions  due  to  centralized  data  base  updating  have 
not  been  achieved,  although  the  process  for  generating 
hard  copy  schedule  data  and  other  manual  references  has 
been  streamlined  somewhat  through  the  introduction  of 
word  processing  equipment. 

Agent  Job  Satisfaction;  All  data  collected  during  the 
evaluation  on  the  subject  of  AIDS'  influence  on  agent 
job  satisfaction  point  to  the  fact  that  AIDS  was  well- 
received  by  agents.  In  the  before-implementation  survey, 

73%  of  agents  responding  chose  "excellent"  or  "good"  to 
describe  their  job  satisfaction;  in  the  after- implementation 
survey  this  percentage  had  risen  to  85%.  On  the  after- 
implementation questionnaire,  agents  were  asked  to  rate 
the  influence  of  AIDS  on  their  job  satisfaction.  To 
this  question,  76%  replied  "positive  influence"  and  the 
remainder  replied  "no  influence".  None  of  the  responding 
agents  responded  "negative  influence". 

Data  Collection;  The  AIDS  software  as  implemented  contains 
a number  of  features  which  enhance  system  usefulness  as 
a data  collection  tool.  These  features  are  routinely 
used  at  present  by  supervisory  and  management  personnel 
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to  monitor  individual  agent  as  well  as  system  perfor- 
mance. Since  implementation,  there  has  been  a dramatic 
increase  in  both  the  quantity  and  quality  of  these 
performance  data.  Unfortunately,  however,  some  items 
are  still  lacking,  because  AIDS  stores  information 
concerning  computer  transactions  only,  and  manual  calls 
remain  undocumented.  Consideration  might  be  given  to 
modifying  system  software  in  such  a way  that  a more 
detailed  version  of  ACD  data  could  be  transmitted  to 
AIDS  and  stored  directly  from  the  ACD  equipment.  In 
this  way,  all  calls  would  be  fully  documented,  and  super 
visors  would  be  relieved  of  the  job  of  recording  ACD 
data  manually  from  the  "peg  counter"  apparatus. 
Interdepartmental/Spinoff  Use;  AIDS  implementation  has 
raised  potentials  for  interdepartmental  cooperation 
within  WMATA  on  a scale  not  heretofore  realized. 

Examples  of  cooperative  efforts  which  have  already  been 
achieved  are  as  follows: 

The  WMATA  Scheduling  Department  had  benefited  from 
standardization  and  storage  of  its  data  on  word 
processing  equipment.  While  this  standardization 
and  storage  effort  was  a necessary  expenditure  for 
the  establishment  of  the  AIDS  transit  data  base, 
the  Scheduling  Department  continues  to  enjoy  cost 
savings  each  time  schedules  are  updated.  Since 
this  process  is  ongoing  these  savings  will  continue 
indefinitely. 

The  AIDS  transit  data  base  was  used  as  a resource 
by  the  WMATA  Planning  Department  in  the  recent 
implementation  of  the  RUCUS  run-cutting  and  driver 
scheduling  system.  It  is  estimated  that  having 
this  transit  data  fully  organized  saved  WMATA  over 
$50,000  in  implementation  costs  for  RUCUS. 
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Examples  of  potential  future  benefits  of  AIDS  are  as  follows: 
The  WMATA  Planning  Department  could  make  use  of  the 
AIDS  Daily  Log  File  as  a planning  rescurce.  For 
example,  origins  and  destinations  of  all  itinerary 
calls  could  be  superimposed  against  a system  route 
map  to  determine  whether  service  is  adequate  in 
areas  with  1-  igh  information  demand.  Frequency  of 
demand  for  information  on  certain  bus  routes  could 
also  be  compared  with  frequency  of  service  on  these 
routes. 

The  WMATA  Bus  Stop  Department  could  make  use  of 
AIDS  in  its  ongoing  program  of  providing  infor- 
mation on  MetroBus  Stops.  AIDS  data  can  be  used  to 
document  which  bus  routes  pass  by  a given  stop. 

The  WMATA  Accounting  and  Finance  Department  could 
use  the  AIDS  transit  data  base  as  a resource  in 
calculating  the  number  of  passenger  route-miles 
provided  by  WMATA  to  each  of  the  numerous  local 
jurisdictions  in  the  service  area.  Using  the  data 
base  in  this  fashion  would  greatly  simplify  these 
calculations,  which  form  one  basis  for  WMATA ' s 
operating  subsidy  assessments.  Given  alternative 
assumptions  on  route  and  service  level  changes,  the 
data  base  can  a] so  be  used  as  a financial  forecasting 
tool  to  show  the  effects  of  proposed  service  changes. 

AIDS  implementation  has  also  raised  the  potential  for 

other  non-WMATA  agencies  to  make  use  of  the  system. 

Examples  of  such  "spinoff"  possibilities  are: 

Use  of  the  geographic  data  base  by  the  Federal 
Government  or  other  jurisdictions  as  a resource  in 
shared-r idership  or  car  pool  assignment  pir.grams. 
Provision  of  the  geographic  data  base  as  a planning 
resource  to  local  governments.  It  is  generally 
recognized  that  this  data  base  is  the  most  accurate 
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street  address  directory  of  the  Washington,  D.C. 
metropolitan  area  currently  in  existence.  The  D.C. 
Council  of  Governments  has  already  expressed  a 
willingness  to  pay  for  use  of  this  geographical 
data  bas'e. 

Provision  of  remote  AIDS  terminals  at  local  transpor- 
tation centers  for  use  by  the  general  public.  Such 
centers  might  include  the  major  METRO  stations. 

National  and  Dulles  Airports,  local  universities, 
etc. 

Thus,  the  implementation  of  AIDS  and  the  concomitant 
establishment  of  its  various  data  bases  have  created  a 
significant  resource  which  has  a strong  potential  for 
serving  a number  of  users  besides  WMATA  information 
operators  and  callers.  The  overall  financial  justifica- 
tion of  the  system  is  therefore  far  greater  than  merely 
productivity  increases  and  cost  savings  within  the  WMATA 
Office  of  Marketing.  According  to  the  AIDS  Program 
Manager,  the  benefits  created  by  AIDS  have  already  greatly 
outweighed  its  overall  cost  to  the  Authority. 

In  conclusion,  while  the  AIDS  system  is  not  without 
faults,  it  is  fair  to  say  that  it  has  already  achieved  some 
of  its  intended  benefits.  The  process  of  system  enhancement 
and  improvement  continues  with  the  support  and  confidence  of 
WMATA  management  and  system  users.  Bringing  the  system  to 
its  present  status  required  dedication  and  hard  work  on  the 
part  of  system  designers,  WMATA  management,  and  the  information 
operators.  One  agent's  final  comment  on  the  After-Implementation 
Agent  Questionnaire  very  accurately  summarizes  the  prevailing 
sentiment  about  AIDS.  "The  system  has  proven  to  me  to  be  a 
very  useful  mechanism.  The  only  improvements  I would  suggest 
would  be  faster  response  times  and  sometimes  not  giving  so 
mai?y  alternatives.  Otherwise,  Bravo  Machinel!!" 
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APPENDIX  A 


REPORT  OF  NEW  TECHNOLOGY 
WMATA  - AIDS  EVALUATION 


This  report  documents  a series  of  data  collection 
efforts  undertaken  by  Wilson  Hill  Associates  at  the 
Telephone  Information  Center  of  the  Washington,  DC 
Metropolitan  Area  Transit  Authority  (WMATA)  during  1980 
and  1981.  The  focus  of  these  efforts  was  a socio-economic 
evaluation  of  the  impacts  of  automated  transit  information 
systems  (ATIS)  technology,  in  which  state-of-the-art 
computer  systems  are  introduced  for  data  retrieval  in 
an  environment  where  this  retrieval  was  heretofore 
accomplished  manually  by  trained  telephone  operators. 

For  the  first  time,  a before-after  implementation  design 
was  employed  to  collect  information  on  the  impacts  of 
ATIS  technology.  Key  variables  of  interest  in  this 
assessment  were  operator  call  productivity,  operator 
job  satisfaction  and  working  attitudes,  changes  in  jobstyle 
and  center  operations  wrought  by  automation,  accuracy  and 
consistency  of  operator  information,  and  cost  savings 
attributable  to  automation. 
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